


ED UCA T 10 NA L 
PSYCHOLOGIC 

MEASUREMENT 


A QUARTERLY JOURNAL D [VOTED TO THE DEVELOPMENT l 
APPLICATION OE MEASURES OP INDIVIDUAL DIFFERENT! 




. ,1 ,. ;m INSTITUTE OF [ U0»Tn.» 
L&lURY & locumeitvwume 

Unit (N.C.E.R T ) 1944 

Acd. 

Dote 




t. 3-^7 


1 


iv m 


y\ 


SMALL- PRINT 4Ptl 




■AN INTEGRAL PART OF 



»f student ,s< 

pwrvo education 

irtd high achooll ha Vo develop 

fit doriving certain skills, l 

writing by doing in the dull 
^g£|n/ Testimony based upon 
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students in such wortb-while experiences. ResJn^tji ' 
cuauxi to thoee student scuvitiu sod. organ 
instructional nature* we observe on inoat^cgr 
dependence upon mast advertising appeals^ 
mouth, printed bulletins, posters, aijd ^stuij 
Notshle-eactpooni an observed in such iiurtir 
College, Stephens College, mid G^pigu^ 
i Rather does the writer refer especially tt» un ty 
to ■ large meat, those who transfer frtffr^ 
Outstanding records of leadership tend 
jOpjMttuakiar while other students, inscrt;iye-iaji 
,wfio lteil'MW experiences, frequently 'trtevitai 
- lines. ( foceaptions to this genejidr" 
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droa tfr. aw t» 1» bnurt d«i « Iw ort nh i"<»- 

«ty af speech, and «ometunci extensive shyne* which may «em 
from myriad cause* Y« niany succea.ful piruc.panu and 
mo leaden may exhibit theie same characteriitja What- 
ever the Inhibitors of desired participation, the prevalence of 
non-participation mdtcwes a serious problem for many aw- 
dwi not only from the etandporot of mental hypenc bat* 
equally ao from the pomt of the univemty’a ceaponnWity for 
UiIium youth in community and nrixenship^ responsibilities 
- and even in the everyday living and working with othcm with- 
out faction- If publicity alone does not produce participation 
' of tfcnae who deairt and need euch experience*, then what can 
be dbno to guide student* with n*P«t to thU phune'of perianal 
devdopmenrl Parenthetically, it a no email cask to identify 
by name *nd person those who do not partlopate. Since they 
ait not pbywcally present at atudoit event* and «[ain, 
nioana must be employed to personalia and individualize wo* 

Numerous attempt* have been nude Jn the Unfvemty of 
Minnesota w evolve % program which would facilitate the in- 
duction of students into activities of interest to then. In 
dreK attempt* there was no thought of hlgh-presminng ^ stu- 
dent* without regard to personal preferences of soma for nor- 
mally reametnd participation. Rather it wm thought that 
publicity regarding the wide range of avalable opportunities 
' Efld the Increasing of this range would ipw facto attract »U 
itudentt who desired to participate. Our openwco did not 
completely Verify such an aisumption, Even the development 
of a comprehpwe rad low-cmt program of recreation in the 
nn^ Coffman Memorial Student Union, while unusually iuc- 
cniful m JaettMlng pirtkipidoti b recreational affaire, ha* 
fcot solved the problems ol an appreciable number of students 
fa*, exptW* a desire to join organization* but who do not 
letootu) tn man invitations of the publicity type. 

A brief review of other program development! in this Um- 
tereiiy will «we u background for the most recant innovation. 
Tn 1930 student leaders petitioned the late President Coffman 
to appoint a committee, with the writer ai chairman, to atudy 


in a comprehensive way the social adjustment problems oi 
students. Prior to this time an informal group of students and 
staff mombere had constituted itself a committee to coordinate 
the many duplicating and competing serial programs, chiefly 
those of a dance nature. Under the leadership of the Univer- 
sity YMCA and YWCA a apecial “Fortnightly” program was 
developed, designed to induct the socially umid and awkward 
into callage recreational programs. A balanced program of 
non-dancing games and beginner's lessons proved to he very 
effective until its popularity attracted those who were already 
well versed m social savoir fairs, thus crowding out than leu 
adept, Interesting vanety In the standardized pattern or 
couple-dancing was introduced by the department of physical; 
education by means of square dances, outing activities, and' 
intra-mural sports, 

The committee** survey stimulated an expansion of dip,? 
social -recreational program of the Student Union and led to 
the appointment of a Social Coordinating Committee which 
continues its attempts to plan n community program for thfe 
University as a whole. This committee, working with "the. 
Calendar Committee of die All-Urn verity Student Council^ 
Baa achieved some success with respect to the problem of coriSI 
fflcts in functions, programs and schedules of social activities, ' 

At President Coffman’* suggestion the Aaiiitant Dean ol u 
Men developed a Leadership Course to coordinate and improve' * 
the efforts of student leaders in their conduct nnd leadership of ij 
organizational affaire. Little effect was noticeable with respect , 
to increasing social participation but this group did achieve * 
some success through its Social Etiquette or Skills Coune. 

The YMCA and YWCA achieved notable success in the 
development of small chibs and discussion groups in which 
social adjustment was facilitated through intimacy of contacts’ 
and concentration of program events upon specific common,^ 
interests. The YWCA groups are impermanent discussion 
groups while the YMCA groups are fraternal but non-Greek, 
letter clubs of a continuing type. 

The above brief review indicates some of the expanded 
recreatidhal facilities Failure to participate was noc caused 
by scarcity of opportunities But recreation is only one 6fi the 
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many types of roaal adjustment opportunities 1 In » normal 
-.peace-time yearorn three hundred student organization* spon- 
sor many types of activities in addition to the recreational 
program conducted by each organization largely lor in own 
members Tbw three hundred oigtmutiQDi cover the full 
range found On any campus, and provide neb opportunities tor 
social adjustment, personal development and atuenihip train- 
" (ng. The Intensity of appeal, issued by these otganiutiona 
» for student workere and participants and the relatively ■ m J u 
^ attendance achieved at their affairs probably indicate both *e 
- fceflfectwenese of the advertising methods oled and the inertia 
of timidity non-pauiapinng student*- Each year during 
Trtshman Week new students are harangued and cajoled with 
i«xrd to "coming out" for activities “after gening your study 

habit* developed " , ,, 

Clearly sludSrts at Minnesota ire presented with oppor- 
■ tunifKt and rather full information about them Should one 
, 'rahcludd that all those who desire to participate respond to 
* there mate appeal* and that those who remain away ’do so 
" Sreuib desire nr are satufactonly adjusted in aocial relation- 
ship*! The iff}2 siuvey of participation (to hive bfxn re- 
^ toSrtid IP 1942V showed otherwise Many ntw-participanta 
reported laekqf knowledge of how in begin or indicated n«»- 
- tancy in reding the flretitep— perhaps mats advertizing meth- 
yl MvoreacMd UieTr Hflpbmum useJultjere and must be«>- 
, 'pifffitftttd by ojh*f -more bi'diviJualiaod methods. Pernajw 
1 fa-peed Ja«olM»t6 hetf the experience of group workers in 
wmraiinfty agenawi tfith ftgpct to the Special need for pt> 
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As of Jufy 1, 1943, certain functions related to the lisped 
vision of student pcguttatjoni and zcnviti** were regrouped 
administratively 5n a new Student Activities Bureau within the 
Office of the Dean, of Students. The following functions were 
aiaigned to thd Burwu'ajtaff 
l Ad 
Sit 

S 


conuuuii|Ljf >1“ ■ 

Jbwibred knd indvnduillud induction of new members into 
lit*,* Peibapt we jo MinfieTOta will find that, 
^JSVn bwp solved exhort, but an egt^tuive 
of the literature and petwnil hbsetvztinnf jndicatee 
loftiqeijta- mntcate . 


3. Cooperation with faculty advisers « steaent 
dnvelopment in studme yrnniziuonal leaders 
standing of group and in dividual 
and interest* of the man. 

4 Financial supervision of 
5. Cbunaeilnf individual atudenti regarding th» ,c ft 
pragma in extncunicular acaviHa, organza tfair. 

The bureau's staff ia composed pf bod) meA-*ad women an 
includes two graduate assiitjams This staff will deal with the 1 
organizational and activity affairs of both men and woman 
thereby differing markedly from die situation on many cam' 
puses where separate offlpes often make for difficulties wi 
respect to affaire organized for both men and women. In am ^ 
a middle western state university as Minnesota, die nunifal 
or joint aides' has markedly increased In die past t 
decades. Separate student organ iutiona for women will, how 
ever, he continual and the special interests and problems 
women will not be ignored. 

■ flif fln£ apd fourth function* listed above need no deKn 
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tfoa in thia paper The second function constitutes the Uni- 
versity's equivalent of community group work in * social 
agency The it iff tnemben lerve both u group-work luper- 
viaon and program advisers, and itudent leaden and faculty 
advuen eerve u volunteer group leaden The Bureau's staff 
serves m advuory and iup«rvj»ory capacities and not is admin- 
istrators except in those cases when au organization's program 
is of such s nature aj to jeopardise the University’ ■ public 
relationships. Experience indicates dearly that such instances 
occur infrequently when the relationships between student 
leaden and staff members are characterised by the term ad- 
visory m opposed to admin utrarive, The staff serve* as a 
resource of suggestions regarding new programs and event* 
when student leaden and faculty advuen seek assistance. 
Without such new ideas, organizational programs sometimes 
tend to become monotonously stereotyped and fa3 to attract 
student participants. A further ph«o of program advising 
involves the coordination and eommtyiity^wide planning of the 
-> f^Veral Jvmdred organization! in order to mini- 

the .undesirable conflicts, program and personal, which 
often aHse as a result of failure to doir in farms bon and sched- 
ules of events 

The fifth function, personal counseling, calls for some de- 
scription As has been the case on most campuses, many at- 
tempts have been made at Minnesota to deal effectively with 
the need for assuring students to choose extra-curricular and 
recreational experiences m Ime with personal needs as outlined 
m preceding paragraphs, For the most part there attempts 
have taken the form of dispensing information about activity 
opportunities through the madurni 3f bulletins, poster*, "Ac- 
tmtwti Pay” during Freshman Week, student counselors, and 
faculty counselors. Our experience has indicated that tins 
method of counseling through activity information is probably 
no more effective than is the corresponding method of dispens- 
ing occupational information in the field of vocational guidance. 
Some students need only information about opportunities id 
order to choose experiences which prove to be valuable and 
effective. But many other students need both Information 
and personal coanseUng. Just as the past decade it Minnesota 


has been devoted to the developing of both clinical and faculty 
counseling programs in the held of educational and vocational 
guidance, it is expected that the present decade will be devoted 
to the development of counseling programs in the areas of stu- 
dent finances, discipline, and social adjustment through par- 
ticipation in extra-curricular activities Descriptions of the 
evolving program! concerned with financial and disciplinary 
counseling are in process of preparation, 

While the precise dimensions to be taken and the techniques 
and procedures to be used m the projected activity counseling 
are not now known, our experience to date indicates certain 
things ic will not become ARhough It mint be closely coordi- 
nated with mental hygiene counseling provided hy psychiatrucs 
and psychologists and with clinical counseling regarding edu- 
cational and vocational problems, activity enameling will 
differ from these m that it will extend beyond group therapy 
for serious maladjustments and beyond the measurement of 
aptitudes and interests. It will be as muah concerned with 
‘'normal" students as with those in senoui emotional difficul- 
ties, As now projected, activity counseling will provide assis- 
tance to students m the transition from the simpler society of 
home communities and high schools to the more complex 
society of adolescents which we calls university In this sense 
the Activities Bureau wQl be an agency (but not the only one) 
for the induction of new members into the college society For 
those students who are able to make their own induction transi- 
tion, it will perhaps be unneeded Biit for the large number 
of students who experience difficulties in taking this step, it 
will offer personalized services to supplement the mass methods 
used at present. Such a semco may be unnecessary on many 
campuses but it appears to be needed in this particular type 
of university, It should be emphasized that this typo of a cav- 
ity counseling is supplementary to many other types of ser- 
vices, some of which are formally organized and others of which 
operate informally It ahould also bo emphasized that par- 
ticipation in activities is conceived as only one of several media 
for achieving that degree and type of social adjustment which a 
particular student may desire and need There Is no thought 
of standardizing either the means or the en^s of personal dev<J 
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opment in the are* of social relationships, but rather a type of 
assistance which will aid the student in selecting appropriate 
means to achieve what he conceives to be desirable and satisfy- 
ing personal objectives. Other media for achieving social 
adjustment will bo utilized, including both organized and 
unorganized recreation and personal contacts, attendance at 
ax well as participation in planning and conducting activ'ues, 
rendentinl l.vmg rxpenrnces, rone nlir nnahipi, anil c'ait- 
room and perwna] conracii wii'i siuc'crts ara teachers 

While it may spuear in i l e reid«r of descriptions sheet tins 
ahd other phases of M.’iiiesuii'i penonnri program that mire 
is a highly compartmentalized organization, such is not tho 
case. In a large university, specialization in peisonnel func- 
tions u not only possible but it is also necessary from an admin- 
istrative point of view if the large number of anidants ara to be 
served While such specialization, with its attendant adminis- 
trative departmentalization, does introduce difficulties of ad- 
ministrative and program coordination, discussed elsewhere,* 
the resulting improvement* m effectiveness of personnel sera 
vices is a worth-while gain Incidentally, there specialists sup- 
, plement but do not supplant faculty counselors, who serve 
*‘Sjfiprive|y w “general pracubdnora" in counseling 
,( Activities Bureau, as is now true of other personnel 

Apartments, serves as a focal point of a personnel function 
but not a* an exclusive monopolizer of a function The Coun- 
OdmgjBureau has served u a, pace setter for faculty counselors, 
actually bdplbjf %a increase faculty participation in counseling, 
which tow takes place literally in every building on the campus 
In similar mam rn: is expected that d e new Acr./i* ea Dureaii 
will provide leadership in enlisting cne fjenlty counselors in the 
use o' activities ai ons inenis to furrher person si ilrvelnpne' t 
of students k n hoped that through Uij, nrogrsm, vriulern' 
organization* and activities will be viewed by more teachers as 
5 paving worth -while educational and profesriora! values rather 
diafl merely ■ distracting effect upon students’ Intellectual 
SffbCts. * 
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The beginning of s program of evaluation is being oat up 
with the establishment of the Bureau For the first ynsr, in 
addition to those students, who are referred or come voluntarily 
for counseling, a special group of several hundred students from 
the College of Education is being invited to uae the Bureau's 
facilities. Thu group was selected because of the special 
motivation to he derived from the fact that effective profes- 
sional adjustment of teacher* is dependent to a large extent 
upon adequate Social adjustment. Follow-up studies of these 
students will be made with respect to the detectable outcome* 
of participation Comparisons will be made between this ex- 
perimental group and other* composed of students who make * 
adjustments unaided by counselor* and those who fail to par- 
ticipate An unusual method of identifying students who mey 
need personalized and individualized counseling was developed 
as a phase of tho required speech examination of all new stu- 
dents conducted by the Speech Clinic. The Qimc’s staff is 
alert to the possibilities in activities for psnonal adjustment 
and penonality development Such coses are referred to the ’ 
Activities Bureau Tor' personal counseling regarding group-life 
adjustments. It u expected that the first few yean of experi- 
ence will be devoted largely to perfecting counseling proefe- 
durei, delineating types of Adjustment problems, and testing 
cntcria df the outcomes of participation and non-partiripaddn. 
Among the criteria to be mveatigated are persistence of par- 
ticipation, degree and type of change from high school to col-' 
lege, subjective report* of satisfaction or dissatisfaction, allevia- 
tion of associated adjustment problems, and other factors* in 
relation to leadership or followership and type of activity , / 

With the establishment of the Bureau and the initiation, of,' 
an evaluation program, it is expected that ..the student activity 
phase of the Umvenlty*e student personnel program will gradu- r 
ally yield experiences Indicative of its true valucp Not the 
least of these values may be the creation of awareness in[ roun- 
eelon ol the range of social adjustment; problems of students * 
and also an undemanding of die therapeutic effectivmipf 6 £ \ * 
group adjustments as supplementary and coorilinare wittf jtfdi- , “ 
viduilised counseling procedures and prog^ms. ^ 

~ ■ ■ \ ' \ / ^ 
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PREDICTION OF SCHOLASTIC SUCCESS IN COL- 
LEGES OF LAW II AN INVESTIGATION OF 
PRE-LAW GRADES AND OTHER INDICES 
OF LAW SCHOOL APTITUDE' 


WILLIAM MICHAEL ADAMS 
Uuroii ty d Iowa 


This study was undertaken wiih the objective of discover- 
ing additional indices of aptitude for successful law school 
achievement which might, in conjunction with the legal apti- 
tude test described in an earlier paper (1), provide a more 
adequate bans for advising students who contemplate entering 1 
the College of Law of the University of lows. Specifically, it 
w ii the purpose of this investigation to determine the relative 
Validity of pre-law grade-point averages and scores on the low* 
Qualifying Examination as predictive indices of success in fi rat- 
year jaw at the Univetiity of Iowa. ' 

Because of the variations in grading standards at different 
undergraduate colleges, only those students who had completed 
all their undergraduate work at the University of Iowa and 
who had complete records for one year of work in the College 
of Law were included in tho major portion of this atudy These 
students were selected from entering classes from the yean 1936 
to 1939, inclusive. The number and percentages selected from 
each class are presented in Table 1. 

TTie predictive indite# available for the group of 152 stu- 
dents selected included grade-point avenges for pre-law work, 
and scores cm the 1 own 0ua%»ivj Examination. The Iowa 
Qualifying Exammatwn, which had been administered to all 
die members of this group as entering freshmen in the liberal 
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arts college, consisted of the Iowa High School Content Exam i- | 
notion, loaa English. Training Examination, Iowa Math*- | 
mattes A-ptnud a Test, and the Iowa Silent Reading Test A f 
composite score, consisting of a weighted total of the raw scores | 
an the four examinations, had been computed The raw scores 
on the individual sub-testa of the bacteiy and the composite 
score had been converted to percentiles and were available in 
this rorm It whs assumed that these percentiles were equiva. 
lent from year to year and, for purposes of computation, the 
percentiles were converted to linear scores by use of the modi- 
ficacion of Hull's table prepared by Guilford (3). 

TABLE I 

A'uei&rr mad Pmtmlmir oi Student) Bdaeuijnwi Psnow 
FmSwimn CUuu ta |A» CoUtf) o / Lot) 


Year 

Total awoUcA* 

Tool idecnd 

Panataja 


as 

J3 

3852 


96 






3939 

1939 , 

107 


3925 

Total . 

. 387 

152 




* Tho tali) number carolled la each data included ittuUnta who con 
prelaw work at other InuiniUoM is wMi or is part, time who wnh 
norm of the jnr, and thoce refrttared u freAnwn for mom riian one 71 


On the basis of findings of similar studies made m other 
schools of law by Crawford and Gorham (2), Husband (5), 
and Jacobs (6) ( ic waa decided that it might he profitable to 
make separate studies of the predictive value of the total pre- 
law grade-point averages, the junior year pre-law grade-point 
averages, the pre-law grade-point averages In social science*, 
and the prelaw grade-point averages in physical sciences, as 
well as of the scores on the Iowa Qualifying Examination (at 
this group 

The available criterion of success in firet-year law consisted 
of a weighted total of the numerical grades assigned in each law 
course completed by the student during his freshman year in 
the college of law These weighted total grades were convened 
to ranks for each class, and the ranks were convened to linear 
scores by the method suggested by Hull (4) These linear 
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icons constituted the actual criterion measures utilized in the 
correlational analysis 

The performance of the criterion group of J52 students in 
the aptitude testa and pre-law areas studied is summarized in 
Table 2. 

TABLES 


fnmjhmlMitEmmiluib* 

1 Ganpoon ware . . 

152 

3197 

16J2 

LIhu imn 10— S3 
Facade 

Linear Cm 2l-W 
Ptomatk 7-» 
Ltocat aew* 7-9S 
Parcaptih LW 

SSJrST* 'SS 

2. ttlgt SdLool Cmmi . 

152 

H9I 

14. JV 

3, ifa4s«alkf Aftitodr . 

m 

3452 

1423 

4, JTofOrfc TmmUf •• ,, 

1S2 

5151 


1 Sflrei KmUmt • . .. 

in 

STJOfl 

16J0 

Uwr • care 11-W 
PunfltUt 2-99 

Tool prelaw UFA., , 

Hp, n. atolnO F A. 
Total prelaw Q. P A . 

in 

m 

I st 

!» 

lil 

7*1 

if, 

JOI 

353 

.*90 

ft 

155-334 

1 17-4 JB 

ISU 

1J2-394 

Jr w pre-law O.P A. 

Totil f»lt« a P A . , 

361" 

ot 


D 

1,17-4X0 

1M435 


* A wippIsauMiir atudy tea* mil 4 361 nultnd 
T A toeofld nppbiiMntvr itoif wu mad* of the rw of &J radio t*. 

The mean linear scores of tbit group on the Iowa Qualifying 
Examination and its tub-tests Were definitely superior to the 
scores of liberal arts freshmen as a whole, hut the range was 
very wide, varying from scores corresponding to the fifth to the 

ninety-ninth percentile in the composite irort- Thq mean total 
pre-law grade-point average* of 2J5B for this group was also 
Superior to the gride-point average of approximately 220 
achieved by students of the College of Liberal Am u a whole, 
but probably not much superior to that of liberal arts senior*. 
Again the range U wide, extending from a grade-point avenge 
of 1 55 w not of 3 94 Fmm there dirJ *r would appear that 

the typical University of Tow* iradem who rnieri the College 
rfbes i Iowa is definite) »upeno r , ■■ a freshman, cn the few* 

, * *• to " * 
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Qaahiytng Eteammation and maintains a somewhat better than 
average record in his liberal arts work. 

Inspection of Table 3 reveals that * moderate correlation of 
,47 wu found between the composite scores of the Iowa Q noti- 
fying Examination and the criterion of success in first-year law 
For die 152 students who had received all then pre-law training 
at the University of Iowa A multiple correlation of only iQ 
was obtained between the criterion end an optimum combina- 
tion of the scores on the four sub-testa of the Qudlifying Exami- 
nation lor this group. Of the indices studied for this criterion 


, TABLES 

CwnJatam el (A* Pr4chw fn4c*i frith lAa Cnterfoft 


tope Oumtifytv EmmlMtb*. 
.Mima «(*• 


al pre-hir mire 

‘ ‘ 

Phyral irlanrr , 
Tout pre4*w wock . 
Junfcr yor .... 

Thai prelaw work 


ft 

JH 

AS2 


i to den a who hid tek*» 


group, the total predaw grade-point overage appears to comj 
sutute the most valid single index of success m fint-year UrpMj 
correlating 67 with the criterion. A multiple correlation sO 
only Si was obtained between the criterion and an optimiiBi| 
combination of total pre-law grade-point averages and coiU^| 
posite scores on the Qualifying Examination. Apparently adrjf 
ditto n of this latter indent adds Uttlnto the predictive value 
tonl pre-law grades. 

Since it was found that, over the period of the study, 
dents entering the University of Iowa College of Law had < 
pic ted nil or part of their undergraduate training at 85 diffoi 
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institutions, supplementary studies were made of available pre- 
dictive indices for two different groups The lint of these 
Studies, baaed on the undergraduate transcripts of 361 students, 
regardless of the institution in which they had taken their pre- 
law work, yielded a correlation or 56 between the criterion of 
success in fint-year law and total pre-law grade-point averages 
and a correlation of 55 between the same criterion and junior 
year pre-law grade-point averages. A second supplementary 
study, based on the undergraduate transcripts of 83 students 
who had completed all their pre-law work at institutions other 
than the Univemty of Iowa, yielded a correlation of only .45 
between total pre-law grade-point averages and the criterion 
of first-year law Behoof performance. It would appear, there- 
fore, that total undergraduate grade-point average! do not con- 
stitute a very effective index of scholastic success m law for 
students who enter the College of Law of the University of Iowa 
From other undergraduate institutions. 

lit order to evaluate the possible efficacy of requiring a 
Bachelor’s degree as one of the criteria for adminum to the 
College of Law of the University of lows, the first-year law 
grades of 212 students who had entered the lew school prior 
to obtaining their Bachelor’s degree* were compared with the 
corresponding grades for 149 students who had received their 
degrees before entering the law school. A critical ratio of 2 13, 
significant sc approximately the 2 per cent level of confidence, 
wa* found between the means of the linear criterion scores for 
the two groups, indicating chat the former group was actually 
superior in first-year law school performance to the group who 
had received their Bachelor’s degrees before entering the Jaw 
school. Thu finding is in agreement with the results of similar 
studies ui several other law schools. A further study of these 
two groups revealed that the former group was slightly, al- 
though not significantly, superior to the latter group with re- 
spect to total pre-law grade-point averages. Apparently selec- 
tive non-intellective factors such at motivation, chronological 
age. and continuity of college attendance, as well as scholastic 
aptitude, must be Invoked to account for the superiority of the 
students who entered l«w school after only three years of pre- 
law preparation 
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Conclusions 

On the basts of the findings of tho studies presented in this 
and a preceding article (1),> the following conclusions seem 
justified; 

1. The experimental edition of the lows Legal Aptitude 
Test appears to show considerable promise as a predictive index 
of scholastic success in first-year law for populations similar to 
those included in this preliminary investigation, as evidenced 
by correlation coefficient! of from .48 through 76 between total 
score and first-year law achievement in five colleges Further 
experimentation with this test, involving larger criterion groups 
and utilizing more rigorous statistical techniques, should even- 
tually result in a much shorter and more valid final edition. 

2 Of the predictive indices, other chan the legal aptitude 
test, studied in the present investigation, total pre-law grade- 
pome averages appear to constitute the best single Index of 
success in first-year law for students entering the College of 
Law of the University of Iowa from the undergraduate division 
of this institution The validity of this index is considerably 
higher for students who have taken all their pre-law training 
it the University of Iowa than for students who have taken 
part of their undergraduate training elsewhere. For students 
who have taken all their pre-law work at ether undergraduate 
institutions, tho predictive value of pre-law grades alone is too 
low to be of much value for individual prediction 

3 Composite and sub-test scores on the lotoa Qualifying 
Examination do not seem to be sufficiently valid indices of 
scholastic success m first-year law to be very useful for pur- 
poses of individual prediction at the University of Iowa. Com- 
bination of these indices with total pre-law grad e-pom t aver- 
ages by the method of multiple correlation does not materially 
increase the predictive efficiency of this latter index 

4. In a comparative study of the relative scholastic success 
in law of students who entered the College of Law of the Uni- 
versity of Iowa after only three years of undergraduate prepa- 
ration and students who received their Bachelor’s degrees prior 
to entrance, no evidence wai found to indicate the desirability 
of requiring college graduation as a prerequisite for admission 
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to tho college of law insofar as mholaotrc achievement in fi rat- 
year law was concerned However, it should be remembered 
th« the requirement of a college degree might be fully justified 
on other bases not investigated m this study 

5 A properly weighted combination of scores on the lotoa 
Legal Aptitude Test and totil pre-law grade-point averages 
appears to constitute the best quantitative index of success in 
first-year law at the University of Iowa available at present, 
as evidenced by a multiple correlation of .77. It is possible 
thit the legal aptitude test scores alone will yield the best pre- 
diction for students who have taken most of all of their pre-law 
training at institutions other chan the Univemty of Iowa. 

6. Improvement of our ability to predict scholastic success 
in law will probably result from experimentation with optimal 
combinations of pre-law grade-point averages, scores oil im- 
proved tests of legal aptitude and, possibly, comprehensive col- 
lege achievement examinations and objective measures of non* 
intellective characteristics as yet not adequately subjected to 
quantification Of these latter, such factors as interests, per- 
sonality, perseverance, health, and personal history data in gen- 
eral are in need of further investigation Further analysis of the 
intellectual functions required in legal studies and the develop- 
ment of more specific testa for their measurement are needed. 
Such studies will be facilitated by all improvements in the relia- 
bility of the criterion of success itself 
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A TECHNIQUE FOR EVALUATING ASSEMBLED 
EVIDENCE OF POTENTIAL LEADER- 
SHIP ABILITY 

G X. MAN BON aid Q L. FREEMAN 
IfatkwtMara Unhenlty 

The rapid expansion of our armed forces and the growth 
of new government agencies has created a shortage of men of 
proved executive talent. Consequently, the personnel expert 
is under considerable pressure to develop methods for selecting 
men of -potential leadership ability In this oak, ic ia essential 
to pool such pertinent information ss |i available. The pres- 
ent report deals with methods used In selecting a group of 
young men with aptitudes for fauainesa leadership It re 
offered ss one approach to a difficult problem of personnel 
■election 

A number of Urge companies with a policy of teaming col- 
lege graduate* for executive positions have been faced with tho 
problem of appropriate selection. Such companies rarely re- 
veal their selection techniques, and consequently there is little 
available information upon the subject. This was the situa- 
tion presented to Northwestern University’s personnel De- 
partment fifteen yean ago when it hid to select from high- 
school seniors throughout tho country, ten young men who 
would be subsidised in a special training course for ^executive 
work in the field of business “ The group, known as The 
Austin Scholars, were to receive a stipend covering the expenses 
of fouryean of college plus a year’s travel abroad As a result 
of this subsidy there was no dearth of applicants, and standard- 
ized procedures had to bo instituted Tor their progressive elimi- 
nation. 

Hie method of selecting Austin Scholars evolyed around 
four interlocking problems. These were (1) making a jpb 
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analyau o( tha duties nntT qualiuti of business leaders, (2) 
securing a population sample likely to contain potential busi- 
ness executives and assembling information relevant to its lead- 
ership qualities, (3) judging upon the baais of assembled evi- 
dence the relative degree to which individuals possessed the 
desired qualities and (4) selection from the top rank* of racer 
judgment upon the basis of personal interview Each of these 
problems will be discussed ui turn. 

I. Job Analyns 

In order to make an intelligent selection of men to be 
trained (or future executive positions, it was first necessary to 
ascertain the special qualities found in persons who were already 
successful on the job Under the auspices of its director. Dr. 
D T. Howard, the Personnel Department of Northwestern 
Univereity prepared suggestions for a functional job analysis 
and submitted these to recognized business leaders for correc- 
tion* and recommendations. The analysis which emerged 
paid attention both to the factors held m common by execu- 
tives and all men of talent and to the differentia of the job, — 
that is, the functions, methods, and abilities which distinguished 
it from other types of work Among the factors common to all 
men or talant primary weight was given to intelligence. The 
differentiating features were less easily defined through study 
of the functions of the business executives These were listed 
*s follows i 

1 The executive must secure and evaluate information 

2. Ho is required to be a planner and work-org.imzer. 

3. He must engage in conferences and communicate ideas 
effectively 

4. He must assume responsibility, making decisions and 
standing by them 

5. He must maintain effective interpersonal contacts and 
have reputation for pereonal integrity with both his 
employees and associates, 

The personal qualities judged most appropriate to the suc- 
cessful execution of the above functions were divided into five 
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classes: (|) physical traits and endowments likely to favor 
effective communication of ideas, (2) mental abilities essential 
to the planning and organization of work, (3) social talents 
involved in the cooperative and directive phases of executive 
performance, (4) character traits contributing co confidence m 
one's integrity, and (5) cultural accomplishments making for 
pereonal distinction, leadership, and prestige. 

II. Sampling Procedure 

With the foregoing analyte* as a guide, the Personnel D*. 
partment next proceeded to secure its sample of high-school 
seniors and to obtain information of likely value in estimating 
the leadership potential of the individuals under consideration, 
On the assumption that high-level Intelligence is a necessary 
starting point for further screening, this was the lint point of 
emphasis Letters were sent to over a thousand high- school 
principals, asking them to post a circular describing the Austin 
Scholarship Program and to recommend their roost promising 
students who met the following qualification*: 

Only thus students will be considered who itai 
tench w their graduating classes m scholastic achie 
conn must be of sound health, able to talk well and 
be loaders among their fellows, and must be respected fa rhar rom- 
muoi t}B* for their moral and psreonil qualities. They must not have 
previously attended college elsewhere, and no person need apply on- 
it “ hii intention to prepare himself for executive work m the 
field of Dunnen 

In the year covered by this report 441 young men signified 
their interest by inquiring about their eligibility for Austin 
Scholarships Of this number 109 did not follow their inquiries 
with formal application and an equal number were found In- 
eligible for rearona ranging from lateness of application to lim- 
ited scholastic requirements. This left 223 young men who 
satisfied all technical requirements They came from 35 dif- 
ferent States and the District of Columbia. A folio compiled 
for each of these applicants earned the following information' 
Form A. Personal history, including photograph, place of 
residence, schools attended. Interests and active 
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ties, outside employment, and future occupa- 
tional plana , 

Form B, Complete record of health from family physi- 


third of the composite "order of meric** hat. This procedure 
of progressive elimination first reduced the list or 223 applicants 
to BQ applicant* and finally to 25 applicants It should also be 


Form C Complete ocholaitk record from high-echoo! 
principal 

Form D Ratuig* on five personality attributes (leader- 
ship, originality, popularity, apeakiog ability, 
and character) secured from three persons well 
acquainted with the applicant. The complete 
rating form, referred to oa Form D, Is grven 

herein 

Form E, Standardized intelligence test {Psychological 
Corporation, Tetl VI) given by high-achoo! 
principal and returned co the Personnel Bureau 
for icoraig. 

Form F. Standardized English placement teat {Colombia 
Research Bureau EnglirK Test) given by higfa- 
rehool principal and returned to the Personnel 
f Bureau for scoring. 

Form G, Correspondence and supplementary letters 
III, Rating Assembled Evidence 

With 223 superior applicants technically eligible for the 10 
Auitip Scholarships, some pioceaa ol further screening had to 
be set Up so that due consideration could be given each case in 
the assignment of relative rank. The technique finally devel- 
oped involved five step*. (1) breakdown of list of candidate* 
into groups of thirty each, in order of completion of applica- 
tion; (2) further breakdown within each group of thirty, so 
that three inter* independently ranked the leadership qualities 
of each candidate relative to a group of ten applicants, (3) 
rejection of twtwhlrd# (approximately) of the total group, 
largely upon evidence from objective testa; (4) terankipg of 
remaining candidates (in groups of ten) by three new raters, 
and (5) combining, for each candidate, the ranking* received 
from the til raters and then selecting for further study by inter- 
view those candidates who composed approximately the upper 
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FORM D— (CwufuMfl 
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noted that as a result of the methods used, each candidate in 
the first elimination senes received a man-to-man companion 
with three different groups of ten candidates each, and that 
thoM who survived to the second elimination senes received 
similar man-to-man companions with three more groups of 
tan candidates each. A random sampling technique was used 
for placing a candidate in the ten-unit groups assigned to ntorv. 
While there was some duplication, the total constitution of 
each group was thus different for each rater By this method 
each applicant was assured of a ranking by each of three men 
(6 for those who survived first elimination) and was compared 
with a number of other candidates, but not in groups so large 
m to moke relative ranking difficult or meaningless The de- 
vice tended to prevent inequalities which might appear if the 


different groups of ten each were, as groups, unequal in ability 
or achievement. It also made for efficient work-organization 
ui the Penonnel Department, since it enabled raters to start 
on the elimination problem before all applications were com- 
pleted and at timea which fitted into their other work-sched- 
ules That the rating method was fairly reliable is indicated 
by the following intereorrelations between raters In the first 
ranking procedures, reliability coefficients between judges were 
as fallows* 1 and 2, r«*+ 70, 1 and 3, r=+^9, 2 and 3, 
*’*+ 73 In the second ranking procedure, reliability coeffi- 
cients between judges were os follows: 4 and 5, rs=+ .54; 4 
and 6, r = + .61; 5 and 6, r ■= + .73 When rater evaluations 
given candidates who survived both competitions wen com- 
pared, the following intereorrelations resulted i 
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The lire of the individual reliability coefficients indicates 
that there were relatively few wide variations in the rating of 
applicants. Examination of the cable shows the usual ten- 
dency for reliability to increase as the final score becomes an 
average of a greater number of ratings A reliability coefficient 
of .91 would indicate that the technique of rating is about as 
consistent as the usual objective examination 

We shall next discuss the qualifications on which the candi- 
dates were compared in the above procedure. There were 14 
of these qualifications and these wore considered one at a time. 
The procedure was to examine the assembled information con- 
tained in the folder of each of the ten members of a given 
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group with respect to the quality under consideration. The 
individual judged to have the most favorable manifestation of 
the quality was arbitrarily given the highest rank of 10, and 
the applicant with the least favorable manifestation a rank 
of 1 The final ranking was the summation of the individual’s 
positions an tire 14 qualities listed below, after the leadenhip 
rank had been given a triple weighting and the character score 
a double weighting. 
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It will be recalled that two further items of information 
have not been used in the above evaluation of applicant!. 
These are the scores on standardized teats of mental alertness 
(Form E) and English (Form F) Although our present 
approach would be to combine such objective test rankings 
with die total rating score, in 1932 the objective tests were 
u *“ eliminate two-thirds of the applicants who composed 
the original sample. That is, those applicants who failed to 
place within the fifth decile’ or better on both testa were ex- 
cluded from further consideration. Such a drastic procedure 
was dict ated largely by B dcaira on the pan of the University 
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that, above all, the men selected should excel in scholarship. 
V/c now question if such an overweighting of the intellectual 
factor waa necessary in view of the high academic record of the 
applicants in high achool 

Reviewing the steps taken in selecting Austin Scholars, we 
have* 

(1) Rating all qualified applicants on fourteen qualities by 
three (Group A) raters. 

(2) Elimination of applicants who did not have a fifth 
decile or better standing on both the psychological and 
English examination 

(3) Re-rating the 14 qualities of applicants who have met 
the standards (imposed in 2) by three new (Group B) 
raters. 

(4) Eliminating the lower two-thirds of applicants on the 
Group B ratings (imposed in 3), and calling the upper 
third (here 25 applicants) for personal interview 

IV The Personal Interview 

It may bo recalled that up to this point, the Personnel De- 
part menu had only indirect contact with applicants and had 
made Its evaluations upon the basis of assembled evidence, It 
was deemed advisable, therefore, to base the final elimination 
upon a personal interview These interviews were handled by 
the Director of the Personnel Department, with some assistance 
from buBiness leaders in the ca>e of certain candidates The 
person conducting the interview did not know the previous 
rank order of the applicants referred to him. Consequently, 
he could obtain unbiased impressions The thing most sought 
In the appraisal interview was evidence of ments) agility, earn- 
estness of purpose and ability to handle oneself effectively in 
a difficult situation. The results of the interview were now 
evaluated against the evidence already assembled on the 25 
candidates, and the final selection of 10 waa discussed in con- 
ferences between the Director of Personnel and the six raters 
who had participated in evaluating evidence. In terms of their 
relative ranking on assembled evidence (Part III), the ten 
■elected stood lit, 4th, 5th, 8th, 9th, 13th, 14th, 16th, 20th, and 

last in fh« group of twenty-five. 
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From a fifty per cent rents of questionnaires sent to scholar 
and .non-scholar groups five yean after their graduation from 
the University, it appeara that the five Austin Scholars who 
replied earn higher salaries, do more supervisory work and 
receive more promotions than do the five non-scholars who 
replied. Averages are given in Table 2, together with Fisher's 
t testa of the significance of tho differences found. 


TABLE l 

Cump *M>% «f dnwii Metr. iflaiir i/ Pttem 8» f tn4 i* i t*d tfnnAss */ 
AtwiiZm/ Lwl fe StXeiart **4 NaZ&tkaUt 




AKhanwWaMnAsIX IwiL 


VI Comment 

It is recognized diet the proposed technique (or selection 
of potential leaden hot received no precise validation. Never- 
theless, Its internal consistency might commend It to busmen 
organizations, rniGtary boards, and government agencies who 
have to select from a large number of applications for a given > 
job. By using a rating form appropriate to the job, the sotting 
of application papers can proceed in an orderly fashion. The 
authors recommend especially that applications be assembled 
m groups of thirty as they are received and that this number be 
broken into three random groupings of 10 each, so that each 
application is evaluated by three different taten in reference to 
a somewhat different sampling. By considering only the top 
third rankings (avenge of three raters) from each group pf 
thirty, a large number of applications can shortly be reduced 
to more workable proportions for a second comparison by a 
fresh group of equally competent rate™ If necessity, the top 
third' rea again be skimmed from the second comparison and 
arranged Car re- nuking a third time. By such a method, the 


V Evaluation of Method 

The personnel worker is naturally interested in the outcome 
of the men selected by our method. Unfortunately, it has not 
been found possible to complete fully the file of each Austin 
Scholar five yean after graduation But the fact that a num- 
ber of Austin “rejects" enrolled in Northwestern University of 
their own accord makes possible certain group comparisons 
between scholars and non-ecbolare The grade-point averages 
for 10 Austin Scholars was 5 73 and for 20 non-scholara 
4.90 This difference approaches significance (GR. 2 1), 
Furthermore, a numerical check of campus activities during 
the four years of college gave the Austin group an average 
count of 12 and the non-Austin group an average count of 7 
activities The difference U not starts crcally significant. 

Such evidence of leadership ability is difficult to evaluate 
and a better test is a comparison of the scholar and non-scholar 
groups in terms of achievement after graduation. We selected 
for this purpose ten mon from both the scholar and the non- 
scholar groups whose post-school records were complete In 
the non-scholar group wa find, some time after graduation, * 
salesman for a meat-packing firm, a research associate for a^ 
peace organization, a minor civil service employee, an auto 
salesman working for hia father, a salesman for a hook concern, 
an accountant, a chemical engineer, a high- school teacher, an 
office assistant in a large grocery chain, and the assistant gen- 
eral manager of a national manufacturing organization. The 
scholar group contains, besides accountants for two largo In- 
dustrial organizations, two specialists in industrial engineering, 
a high-school teacher and five men who have risen to mana- 
gerial authority in large corporations. Included in this most 
successful group of former Austin Scholars is the associate edi- 
tor of a national magazine,, a purchasing agent, an executive 
assistant, a plant superintendent, and an assistant sales man- 
ager. While quantification of such evidence is difficult, it 
would appear that many Austin Scholars have done well in the ' 
bus mew world even daring an economically unsettled decade. 
The above evidence Is influenced by the training given to 
Austin Scholars as well as by the selection method 
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men finally chosen will have been ranked by a number of raters 
relative to several groups of applicants and in reference to a 
fixed ret of critoria 

The authon hold no special claim for tho specific criteria of 
leadership ability uaed ui the present selection procedure. 
Three are probably no better nor worse chan the lists of quali- 
ties now in use elsewhere in the rating of leadership potential 
It should be pointed out that before any such list of qualities 
can he used with precision, it will be necessary to determine the 
degree to which each item discriminates between leaden of 
demonstrated ability and non-leaders. Until then, the relative 
weighting of Items on a leadership rating scale must rest largely 
upon the opinion of the examiner The authors feel that ab- 
stract intelligence was overweighted jn the selection of Austin 
Scholars. The fact that this type of overweighting is chirac- 
tenitic ol most leadership selection procedures is no justifica- 
tion foe its continuance. It seems that personnel workers 
might well give further study to an alternative proposition. 
This proposition is that since leadership v a social quality, it is 
better judged by evidence of effective social Interaction than 
by non-eocnl tests of individual ability. 
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JUDGMENTS IN COUNSELING 1 


RALPH F BERDIE 
Um«nlij of M Imran 


Vocational counselors in student personnel programs at- 
tempt to assist students make judgments about their awn 
capacities, potentialities, and personalities The counselor 
supplies the student with two things Primarily, he gives the 
atudent information about himself and his relationship to his 
surround mgs The student u told haw his various abQities 
and interests, as estimated by teat scores and past achieve- 
ment, compare with the abilities and interests of people In dif- 
ferent jobs, schools, and curricula. The counselor then helps 
in the use of this information so that the student can make a 
sound judgment regarding plans for the future. 

Techniques for estimating abilities and interests have been 
discussed by many writers and summarized by others (1) 
Methods of assisting students m arriving, at judgments have 
been discussed by Williamson, Rogers, and other* (4, 2) Be- 
fore a student can Understand his situation and plan for tho 
future, he must make companions and discriminations In- 
formation that has been analyzed and evaluated results In 
judgments, which in turn lead to decisions, which in their turn 
lead to action. 

An attempt to adopt methods of clinical psychology and 
psychiatry in vocational counseling has led to a difference of 
opinion regarding underlying philosophies and techniques. 
One group of workers emphasizes the judgments that the coun- 
selor must make. Another group expresses the opinion that 
it is not only unnecessary but alio undesirable that the coun- 
selor make judgments, The counselor, according to tho latter 



group, functions only to assist the student in arriving at judg- 
ments and does not provide judgments of his own which the 
student can accept or inject. 

The view of the group deeming the counselor’s judgment 
essential is expressed hy Williamson (4, page 118)* 

It should be repeated (hat m vocations! counseling the counselor 
collects data shout the student'e potentialities and than proceeds to 
compare these data with the requirements of the student's expressed 
occupational choice In this way he strives at a diagnosis of apti- 
tude and ■ judgment as to the wisdom of that choice. This com- 
parison of poUMMkiMj trtik preferences is the important Jtep in 
diagnonrJ Following this step comes the interpret* don to the stu- 
dent of the counselors diagnosis and the cooperative planning of 
next steps. 

Hie judgments studied in this investigation were judgments 
made by counselors about students, and the results are there- 
fore most relevant for those counselors subscribing to the school 
represented by the quotation from Williamson 

When a student seeks vocational counseling, he usually is 
considering one of more possible vocations and he is attempting 
to make a judgment regarding the relative appropriateness of 
the considered vocations before deciding to enter one. The 
large number of factors serving to attract a student to various 
vocations results m one vocation usually being more or less pre- 
ferred over others. Students seldom are considering two or 
more vocations all of which seem of exactly equal attractiveness 
They have preferred vocations and they want to determine if 
they should enter those vocations To assist in this determina- 
tion the counselor and the student organize information based 
upon school grades, aptitude and ability teats, achievement 
testa, interest tests, personality scales, hobbies, and work ex- 
perience. Thu organization usually results in both the coun- 
selor’s and tho student’s making judgments about the appro- 
priate new of vocational choices The counselor’s judgments 
may be the most effective force in molding the student’s 
judgments. 

This study sought to determine how consistent different 
counselors were in making judgments about vocational choices 

* talk i* ib« arista] auto**. 
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and what factors were instrumental in their arriving at these 
judgments. The judgments were all baaed upon the tame 
material, it, case folders, aad the counselors all had gone 
through a senes of training clinics that supposedly supplied 
them with a common knowledge of the relevancy of tho tests 
used and the data collected 


Methods 


Twenty cue folden of pre-college men were selected from 
tho files of the Testing Bureau of tha University of Minnesota. 
The vocational choice of each of these students had been judged 
m the first counseling interview as either appropriate or inap- 
propriate by the counselor. The case notes were then reviewed 
by s case reader who vended the judgment of the ongrntl coun- 
selor A duplicate folder was then prepared for each of the 
cares and all identifying data, such aa usme of student, name 
of father, etc , were removed Hie Following matenal wu con- 
tamed in the folder in this order; 


1 


4. 

5 

6 , 

7 . 


The preliinTrtaty interview report — containing the student's 
naced problem, his high-school record, end observations made 
by the receptionist 

The summary profile of test scores — containing the names of 
the tests, the raw scores, the percentile scores, and the norm 
groups. Tests given ac tha Testing Bureau include tests of 
scholastic aptitude, scholastic achievement, special abilities, 
and personality 

The Strong Vocational Interest Blank profile. Spires were 
recorded for twenty-tight occupational scales and two non- 
oecu^ioofix] scales, matcuUmcy-femlcututy, sod occupational 

The Ruder Preference Record profile, un revised form These 
profiles were not available for all of the twenty cases 
The University Tea nog Bureau Individual Record Farm — a 
quflrtwnnaire covering family k background, work experience, 
rears uon*I activities, occupational plans, health history, ere. 
A short statement of factual information obtained by the 
original counselor In the fust interview with the student. No 
interpretative statements v 


A raring form _ 

The student came* to you and says, “If*you say ’yes,’ I 
will attempt to become a(n) (tiuett occupational choice) If 
you say no, I will not " 

In thu case you are forced to make » judgment. On the 
basis of the contents of thu folder alone, would you say: 

Yre No. . (diedt om) 


the three following statements , 
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L If you think someone might disagree with you on the above 
jud^mui^ what do you think nu bans Tor disagreement 

2. Why do you dunk thle basil insufficient: to changn your 

judgment? 

The folden were then arranged in numerical order and 
given to each of the five counsalon wjtfi the following instruc- 
tions 

This is a necessary step in determining how Judgments regarding 
the appropriateness of students' vocational choices are made apd 
how consistent these judgments are. Will you please read through 
the twenty folden in the order in which they are given to you and 
after you have obtained tho oeoanaiy information is you do in the 
interview or prior to the interview, answer tho questions which are on 
the mimeographed sheets m cadi folder. 

It is impossible to duplicate the exact interview situation without 
the student, but in order to provide ss much ef the interview atmos- 
phere as possible, the date observed end recorded hy the ongmal 
counselor have been included in the folder. Regard these observa- 
tions as factual rather than interpretive, is, they are exact stated 
monte of description rather than what the observer thought Please 
be sure to answer til the questions 

After each counselor had rated the twenty cases, a pro- 
cedure requiting From two to four hours, the folden were 
checked and put in their onguial order New rating forms 
were inserted and the folden were given to the next counselor. 

, Results 

Table 1 shows those cases judged by each counselor as hav- 
ing appropriate vocational choices and those having inappro- 
priate choices The check after the number of a student indi- 
cates that the counselor judged his choice as appropriate. The 
absence of a check indicates that the counselor judged it as* 
inappropriate. 

Of the 100 judgments, fifty-five were in the direction of ap- 
propriateness and forty-five m the direction of uiappropriate- 
ness. Of the twenty students, ten had been Judged by the 
original counselor aa having appropriate vocational choices and 
ten as having inappropriate choices. Eighty-one of the J00 
judgments made by the five counselors agreed with the judg- 
ment previously made by the counselor who originally inter- 
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viewed the student. Analysis of the data m Table l yields a 
tdfrachonc coefficient of .86 between the judgments made by- 
the original counselor and the judgments later made by the 
five counselors. TTie results indicate that in judging the ap- 
propriateness of the vocational choices of groups of students, 
trained counselors are able to agree, 

TABLE 1 


JadnwaiJ °l CoMhirian Ctmimt A oj i*a FowlW 
Cioww aS TutiUy SIiUmwU 



Of the decisions originally made in the direction of appro- 
priate choice, 14 per cent were later judged as inappropriate. 
OF the decisions originally made in the direction of inappro- 
priate choice, 24 per cent were liter j'udged as appropnate. 
This suggests that counselors tend to agree more upon judg- 
ments when students' choices are appropnate than whoa they 
are inappropriate. Counselors are perhaps more Willing to put 
their stamp of approval on the student’s plahs than they are to 


discourage him. If a doubt exists in the mind of a counselor I 
regarding the appropriateness of a student's vocational choice, | 
he fa more apt to decide the choice warrants a try-out than to ! 
decide it does not These generalizations spring from clinics] * | 
experience and the above data tend to substantiate them. J 

The judgments of the majority of counselors (three or - 
more) agreed with the original cla ss ifica tion of appropriateness- ’ 
in appropriateness in eighteen of the twenty cases. The only j 
people who considered ehe choice of case 14642 appropriate ‘ 
were the original counselor in both h» first judgment and later ' 
judgment, and the case reviewer. One counselor besides the ’ 
original counselor and the case reviewer considered the choke 
of case 13964 inappropriate, but three counselors considered it 
appropriate. The results do indicate that in the great major- 
ity of cases, judgments of the five counselor* agreed with the 
decisions reached by the original counselor and the case . 
reviewer 

Fourteen of the twenty cases had originally been inter- 
viewed by counselors who later served as judges in this study. ’*] 
It is doubtful if the original counselor* remembered the ,■ | 

all identifying data were removed at second presentation 
and a period of several montha had elapsed since the original 
interview Of these fourteen judgments, not on* disagreed 
with the original decision. Even those two cases on whom the , 
majority of counselors disagreed with the original counselor 
were consistently judged a second time by the original coun- 
selor. Here is evidence that a period extending from two to 
four months does not alter the judgments a counselor makes 
regarding the appropriateness of vocational choices and that 
counselors tend to arrive at tho same judgment in the inter, 
view ■■ they do by merely reviewing the case data and never 
seeing the student. These are essentially the results found by 
Super and Brophy (3), and it seems safe to generalize from 
them in the area of vocational guidance. 

Up to this point, the analysis of judgments concerning ap- 
propriateness and inappropnateness of vocational choices has 
provided Information regarding the consistency of these judg- 
ments. Little has been shown regarding the actual judgment- 
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making processes themselves, however, and* the facto is con- 
sidered in these judgments have not been discussed, A more 
complete understanding of these judgments will be obtained 
through a careful consideration of some of these cases them- 
selves and thd comments made by the counselors as they were 
making the judgments Limitations of space prevent presen- 
tation of this material for all of the twenty cases 

Cat* number 13755 was a seven reen-y ear-old hoy who grad- 
uated from high school one month after be came to the Test- 
ing Bureau. Hi* occupational choice was chemical engineer- 
ing, hi* father was dead, ho had one eleven-yeawdd brother, 
and he said he would have to work about twenty-four hours 
a week while attending college 
Teat scores were u follows; 
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Strong T*n scores were A on key* fop eheptut, farmer* 
carpenter printer, mathematics and physical science teacher, 
forest service nun; B+ on key* for dentist, engineer, Y.M.CA. 
physical director, social science high-school teacher, office man; 
B on keyi (or production manager and accountant. Mtuen- 
knity^Fetn ini n i ty percentile — 82; Occupational Level jjercantDc 
—2 Percentile scores dn the Kuder Pnftttnt t Rieord werej 
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Scientific, 98, Computational, 30, Musical, 8 ; Artistic, 86; 
Literary, 8; Social Service, 9, Persuasive, 44. 

The following statement of factual information wit ob- 
tained from data contained in the interview notes of the coun- 
selor who originally counseled the student. 

He is a husky, rather ordinary appearing young man He u a 
Uttk etude in Ms approach but very friendly rill lather Is dead and 
ha lives with his mother and younger brother 

He will be totally self-supporting. He has saved about 9100 from 
hi* pspsr route ui rhe lest rnree or four yean and will continue to 
have s morning and evening route whde going to allege 

He nye he cook (us first two yean ui inch school as a joke and as 
compulsory education. Then he realized ns must work harder but 
was unable to fully compensate for his piwwiu loafing He seam* 
well motivated. 

The original counselor and the ease reviewer originally 
decided that the buy*! choice of chemical engineering was ap- 
propriate. In making Hie later judgments, four of the five 
counselor* agreed that this choke was appropnate; one decided 
it was not. 

The dissenting counselor's response to the first question 
concerning the basis for disagreement waa as follows j 

They would *m*« the adequate ability and high -school achieve- 
ment furore, together with the A and Bf scores on tho chemist and 
engineering key on the interest tat. They would then fullest that 
the boy b<e erven a chance ro show what hie could do They would 
■la stress the good tents of background in mathematia and chew 

" in response to the second question, concerning his reasons \ 
for considering the above inadequate to change his judgment, • 
be wrote- ' 

The discrepancy between H.S.H. and A C£. or 0JUo seems tor , 
Indicate undeiwadiievemenc of lone standing. The narrowly Inw^ 
Occupational Lend score and die problem or complete financial sup-- 
port are not hopeful sign* °f waving power The block of ikdled 
trades interests on the Strong Tejl is vitally important, partially 1 
because of their rarity. Jj 

Three of the other counselors mentioned the low high-school $ 
rank but did noc appear to give it enough weight to have thwrtJ 
change their judgments One counselor thought a basis fof^ 
disagreement might be found in the fact that it was too 
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for the fray to make a choice as Bpecific as chemical engineering. 
Two counselors specifically mention the boy's motivation and 
appear to think that the fact that his achievement testa arc 
better than his high-school rank improves his chances in the 
umvcraity, 

One counselor, rating the occupational choice as appro- 
priate, recognized these negative factors 

1 Average maihcmana rat score 

2 Low high-school rank 

3 Low Occupational Level score — "interata evidently imma- 
ture" 

4 Can mathematician key of Strong Tut 

5 Totally eelf-iupporunp 

C Low grades in high-scnool mathematics 

We thus have two counselors who considered and listed the 
tame factors, and yet one decided the choice was inappropriate, 
while the other wrote’ 

The above n strong evidence to question his choice, but because 
of the positive factors which ire also dearly indicated, I d be inclined 
to give him the benefit of the try-out. 

The above counselor's note concerning the Occupational 
Level score gives a suggestion concerning one possible source 
of this disagreement Thu counselor evidently interprets the 
Occupational Level score as an expression of interest maturity 
which therefore might change as the student agcB, while the 
first counselor vaguely interprets it as an index of academic 
motivation 

Only one of the counselors approving of the occupational 
choice mentioned the financial prohlem, and none of them em- 
phasized it as being important The ungmal counselor recog- 
nized the financial problem in the interview and took steps to 
meet n Although he realized the importance of this problem, 
he did not consider that the boy should stay out of engineering 

This boy completed the first two quarters in the Institute of 
1 ethnology with an honor puint ratio of 42. His grades were* 

College Algebra D 

Chemistry C, C 

Composition D, I (incomplr* ” 

Drawing D, D 


He dropped out of school at the end of his second quarter 
and has not returned. 

Case number 13972 was an eighteen-year-old boy who grad- 
uated from high school six months before coming to the Testing 
Bureau and who was completing six months of post-graduate 
work m high school His occupational choice was mechanical 
engineering, hu father was crippled with arthritis and unable 
to work, he bad two younger sistera, sixteen and eleven years 
old, he had work experience drafting on an N YA job, and 
would be totally self-supporting. 

Test scores were as follows* 
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Strong Test scares were A on keys for chemist, production 
manager, fanner, carpenter, printer, mathematics and physical 
science teacher, accountant, office man, B+on keys for engineer, 
forest service man, B on keys for dentist, personnel manager; 
social science high-school teacher, purchasing agent. Maicu- 
Imity-Feminmity percentile — 76, Occupational Level porcentde 
— 22 . 

The following statement of factual information was ob- 
tained from data contained in the interview notes of the coun- 
selor who originally counseled the student 


The boy does not have much money. His father ib disabled be- 
cause of arthritis. The boy has bean working part-time for five 
months on a N YJ\ job doing drafting work and making about 3100 
■ month. He has hod other odd jobs fro in trait to time. 

He is intrigued with mathematics and doesn't know much about 
tha work of an actuary He does not know that the training for 
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thia type of work » essentially through busmen and not through 
mathematics, 

He has applied for a scholarship open to sons and daughters of 
World War veterans 

He is socially retiring. 

The original counselor and the case reviewer decided that 
the boy's choice of mechanical engineering was appropriate 
Id making ihe later judgments, (our of the five counselors 
agreed that this choice was appropriate, one decided it was not 

The dissenting counselor wrote* 

Jins it a loose ludenunt It would actually depend upon a judg- 
ment of which field (Institute of Technology or actuarial) the boy 
wanted most He would be handicapped by lack of chemistry [in 
high school]. 

Three of the other counselors recognize the boy's expressed 
and measured interests m the business field and two of them 
suggest possible combinations of business and engineering. 
One of these two "mildly objects" that the boy hasn't quite the 
penonahey type for executive or management work. Two of 
the counselors indicate t&t the financial problem might serve 
as a basis for someone disagreeing with their judgment of the 
choice as appropriate Both say, however, they think the 
problem can be satisfactorily managed One counselor men- 
tions the low Occupational Level scores but thinks this is over- 
balanced by the excellent achievement in higb school This is 
the same counselor who m the last case interpreted the Occu- 
pational Level score as an index of maturity He is apparently 
using it here as aq indicator of potential achievement! This 
counselor also recognizes the low score on the chemistry test 
but discounts it because the boy has not had chemistiy in high 
school 

In general, there are no fundamental disagreements among 
the counselors regarding this case They all tend to recognize 
that the boy has several potentialities and is a good college risk 
Although one counselor Indicates ho would not encourage the 
occupational choice, he later qualifies this. 

This boy completed the first year in the Institute of Tech- 
nology with an honor point ratio of 2,11. Hu grades werei 
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Algebra A 

Trigonometry A 

Analytic Geometry B 
Chemistry B, B, C 

Composition A, B, B 

Drawing B, B, C 

He is now making satisfactoiy progress in his engineering 
course 

Cate number 14095 was an eighteen -year-old boy who grad- 
uated from high school the same month he came to the Bureau. 
Hi| vocational choice was business, the occupation of hu father, 
Hu test scores were as follows. 
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Strong Test scores were* A on keys for personnel manager, 
real estate salesman; B+ on keys for social science higb-sehool 
teacher, accountant, office man, sales manager, life insurance 
salesman, B on keys for production manager, Y.M.CA. aecre- 
tary, purchasing agent Masculinity nity percentile— 
76; Occupational Level percentile— 66. Percentile score on the 
Kuder Preference Record were: Scientific, 42, Computational, 
79; Musical, 10, Artistic, 7, Literary, 62, Social Service, 60; 
and Persuasive, 99 

The following statement of factual information was ob- 
tained from data contained in the interview notes of the coun- 
selor who originally counseled the student. 
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He u a joung man of short nature, not too neatly droned, haa i 
nthor informal manner to the interview He la rather ordinary la 
Appearance. He would lilts to be self-supporting while grant to 
idiool He hai about $250 raved now and might earn enough before 
tuning at die TJnrveituy to conuaue without working while m 
ichooL Hit father would help him but he diahkea Hiking hie lather 
fwhelp 

The father and mother were divorced about three year* ago, The 
lather ta the present of a busintre. There ire m other children ui 
the family and the boy lives with hu mother and geti along well with 
both parents. 

In high ichool be uya he studied jiut enough to get by but «yi 
he realize* now chat that wu not so smart 

The original counselor and the ease reviewer decided that 
the boy's choice of business was appropriate. In making the 
later judgment, all of the five counselors agreed that this choice 
wu appropriate. Four ol the five said they did not think the 
boy’s chanres of graduating from the Umvenrity were good but 
their attitude u expressed by this counselor. 

I cant lec any bun for disagreement since the fob u waiting for 
him whan he it through with school- However, I do net think heU 
graduate from the Umvenity 

Again in this case there is general agreement among the 
counselors. It is doubtful if the hoy’s plana would have been 
Influenced differentially by the eouruelon at they were all 
ready to approve of hw vocational choice. This case is quite 
clear-cut. 

This boy entered the College of Science, Literature and the 
Arts and at the end of his first quarter had an honor point ratio 
of .00. It is doubtful if his academic "flop" will seriously Inter- 
fere with hii vocational adjustment Hn grades werei 


Composition — cancelled 
Economies D 

Polnicil Science C 
History F 


He did not return to the University after his first quarter. 

Cats number 13835 was an eighteen-year-old boy who came 
to the Testing Bureau the month he graduated from high 
school, His father was a talesman. The boy had oAe younger 
sister The family wai going lo finance hii college education 
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He had no work experience outside of working for his fathu 
and hie vocational choice was advertising. 

The test score* were* 
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Scores on the Strong Interest Tut were, no A ratings, Et- 
on keys for farmer, purchasing agent, real estate salesman; 
B on keys for personnel manager, office man, president of a 
manufacturing concern, MoscuknetyFeminmity percentile-. 
57] Occupational Laid percentile — 40. Parcentild scores on 
the Kuder Preference Record were. Scientific, 57’, Computa- 
tional, 45, Musical, 1, Artistic, 56; Literary, 15, Social Service, 
17; Persuasive, 52 

The following statement of information was obtained from 
the data contained in the interview notes of the counselor who 
originally counseled the student. 

Hem vary immature boy and Ins mother cane with him the 
first day Ho gives the impression of not knowing much about any- 
thing. He doesn’t react favorably to Gcnoral Cbfiegn. 

The original counselor and the cose reviewer originally 
decided the boy's choice of advertising was inappropriate In 
making later judgments, all of the five counselors agreed tbit 
this choice wu inappropriate Three of the j’udges could see no 
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reason why anyone would disagree with them, Two of the 
counselors suggest he nught he able to handle a tower level job 
of advertising, 

They might think he could do a low-grade type of advertising 
(show cud wnimg, -Window displays) which may ho what ht means 
by advert* n* man* . He could never ret Uin degree from the 
adverrltine sequence in the Bonnes* School because m various rea- 
sons. I doubt whether lie’d be much good even at a low level of 
advertising lit view of the poor numpuliuve ikilli and ability to work 
In spibal retiaanships. 

The judges appeared to hava veiy little difficulty in arriv- 
ing at a decision on this case. He entered the General College 
and at the end of the first year took three comprchenrivt ex- 
aminations, receiving a C in Social Civics, an F in Vocational 
Orientation, and an F in Individual Orientation. Hu course 
grades were*. 

Individual Onomarioo C 

Current History B C B 

Government Studies B B 

United States m World civilization B C 

Vocational Orientation D 

Human Biology C C 

Oral Commumuaoa C 

SStt&e _ S 

Human Development C 

Ecowwaei C 

Care number 13964 was a seventeen-year-old boy who came 
to the Bureau the month he graduated from high school. Hi# 
Tather wu a car and truck mechanic and there were two 
younger brothers and a younger sitter In the family He had 
ho unusual work experience and bn vocational choice was 
architecture. Test score* were as follows: 
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Scores on the Interest Test were. A on keys far printer, 
mathematics and physical science teacher; B+ on keys for far- 
mer, personnel manager, accountant, office tnan, B on keys for 
carpenter, forest service man, Y.M.GA. physical director, 
YJ4.CA. secretary, social science high-ochool teacher, Mes- 
cvknity-Fenummty percentile — 30; Occupational Level per- 
centile— 22 Percentile score* on the Kuder Preference Record 
were* Scientific, 50; Computational, 65; Musical, 63; Artist^ 
90, Literary, 35, Social Service, 5; and Periuwive, 30 

The following statement was obtained from the data con- 
tained m ths interview notes of the counselor who originally 
interviewed the student. 


Than is a financial problem here, to same extent. He hu ap- 
plied for N.Y.A. assistance Ha claims hi enjoys art very much and 
has done a great deal of poster work and layout work. He has ■ 
chance for a scholarship sc the Institute of Art but will not accept ft. 

The original counselor and the cose reviewer decided the r 
bay’s choice of architecture was inappropriate. In making the 
later judgments, two of the counselor* agreed that the choice 
was inappropriate, three decided it was not. All the counielon 
rating the choice as appropriate noticed the absence of raoa- , - 
■ured interests but made the following statements ^ 

"No confidence in Strong for architecture/ 1 "Kuder test , 
shows ‘artistic*; haa art akilla, hat necessity ability. Hie'' 4 
Strong test may he in error." "The Strong is against me but/ 
his experience (making posted) is good enough to lead sue td 5 
let him tty it The other interest tut support* hjs choice/* * 
One of the counselors rating the choice inappropriate cited 
»n support the great concentration of skilled trades uiwtettj? 
the low Occupational Level score, the poor home and cultural 
background, find the financial pfoblem. The superior high? j 
school rank was discounted In light of the relatively poor rehopt, ^ 
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the boy attended The other counselor rating the choice u 
inappropriate admitted the boy probably had enough ability 
to get the degree and that the choice might be appropriate from 
rbe irandpomt or job opportunities He added, however. 

He probably 11 not outstanding enough in ability to make his liv- 
ing in this field, which offer* limited opportunities to ill but the very 
exceptional person In view of the lack o[ Strong i measured mu- 
eiti, the low level o! occupational aspiration, Id i»v be should prob- 
ably get mechanical drafting mining or cammerqal art training. 

This counselor, who i« now interpreting the Occupational 
Level score as a measure of occupational aspiration, is the same 
one who previously interpreted it as a measure of interest ma- 
turity! Thu boy did not enter the University 

In this case, the basis for disagreement u quite clear-cut 
Two of the counselors, chiefly on the basis of the interest test, 
would discourage the boy from entering architecture The 
other three would discount the lack of measured interests and 
not discourage the boy. A tendency has also been found for 
some of the counselors to consider some of the keys of the 
Strong Tes t- valid and same not. The factors which help the 
counselors determine which key* are valid have not been 
identified, 

Curs number J40&1 was a nuie teen -year-old boy who came 
o the Bureau one year after graduating from high school He 
nil have to help finance hia college education His vocational 
hoice was metallurgical engineering 
The test scares were as follows 
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Score* on the Strong Test were: A on keys for engineer, 
chemist, Tanner; B+ on no keys, B on keys for architect, mathe- 
matician, printer, mathematics and physical science teacher, 
purchasing agent MasciUlntly-Fwuntnity percentile — 65; 
Occupational Level percentile — 66 Percentile scores pn the 
Kuder Preference Record were* Scientific, 70; Computational, 
91; Musical, 50, Artistic, 55; Literary, 5; Social Service, 9; 
Persuasive, 60, 

The following statement of information waa obtained from 
the data contained in the interview notes of the counselor who 
originally interviewed the student 

He Lvea with his mother and sister, both of whom are working. 
His father is dead. H* applied for ad mm ion last year but was re- 
ferred to the General College. He did not wwh to go there so he did 
not come to school He has in uauaually strong vocational fixation. 

The original counselor and the case reviewer decided the 
boy's choice of metallurgical engineering was inappropriate 
In making the later judgments, three of the counselors agreed 
with this decision while two counselors judged the choice as 
appropriate Two of the counselors rating the choice as inap- 
propriate said they would expect no disagreement. The other 
counselor rating it as inappropriate said that, in the light of 
the boy’s apparent strong drrve for engineering, and "with the 
equivocal results on the Strong's," some might claim he should 
be given an opportunity in engineering. This counselor does 
not expound die mentioned equivocalities of the Strong Test 
He decides, however, that, without an interview, the above 
factors cannot be given much weight. Both counselors racing 
his choice u appropriate recognize the boy's limited abilities 
and one mentions hia relatively woak background in mathe- 
matics and chemistry. One of the counselors writes* 

They would question his ability, His vocational choice has 
changed since he left high school That might help him to work to 
capaaty Then I think he could make it . I would improve his 
motivation, I hope, and if so, he’d make it He Jack* varbahstic 
ikdli. Thii would not be auch a handicap to him m the Institute 
of Technology 
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Thu boy did not register in the Umvemty. 

In this case the bans for disagreement is again quite clear. 
The boy has interests congruent with hu choice but abilities 
and a background that are not encouraging Ail the counselor* 
apparently recognized these factors, but three of them decided 
engineering was not a satisfactory choice, while two of them 
decided to take a chance, not place coo much weight on the 
ability factor, depend on the intereit and motivational factors, 
and let the boy tsy it. 

Summary 0 f Analysis of Judgments 

The preceding discussion has covered much territory and 
many inferences hive been drawn. Inference# drown from the 
six cases discussed and the fourteen remaining cases are not 
conclusions bared upon experimental results but are generaliza- 
lions based upon observations of only a few cases Often they 
are merely chnreal hypotheses that should he investigated. 
Some of them are nothing more chan verbalizations of con- 
ditions generally known to exist but often not explicitly 
recognized 

L In making judgments regarding the appropriateness of 
students’ vocational choices, craned counselor* agree with the 
original judgment 84 per cent of the time. The tetracbonc 
coefficient between judgment# msdo by different counselors 
is .86, 

2. The majority of five trained counselors agreed in making 
judgments regarding the appropriateness of students’ voca- 
tional chokes upon the bins of the cue notes with the decisions 
of the counselor who originally counseled the student and the 
case reviewer 90 per cent of the time 

3. In fourteen out of fourteen case#, the counselor who 
originally counseled the student and judged the appropriate- 
non of his vocational choree at that time made the same judg- 
ment from the unidentified case folder several months later. 

4 Counselors tend to approve of vocational choices uftener 
than they disapprove of them. 

j. The Occupational Level score has been interpreted as a 
measure of interest maturity, aa a measure of motivation, as 
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an index of occupational aspiration, and as a cause for poor 
achievement in high school 

6 Some counselors are reluctant to have a student of high 
ability enter business even if other factors suggest the advis- 
ability of such a choice 

7. When both measured and expressed interests are in busi- 
ness, counselor* will seldom discourage a student from entering 
business. They may disagree on the level of training desired 

8. Inability and interest factors indicate success in an, occu- 
pation, counselor* tend to think meager informational back- 
grounds cgn be compensated for. 

9 The appropriateness of a student's choice cannot easily 
bo agreed upon when the occupational requirements, duties, 
and status are poorly defined 

10 The greater the apparent contradiction between the 
student's test scores, die greater will be the disagreement re- 
garding the appropriateness of hu vocations! choice 

11, Some counselors interpret more rigorously scores on 
some scales ol the Sifong Test than they do scores on other 
scales 

12. Counselors do generalize results of the Strong Test to 
occupations for which there are no keys Often they disagree 
upon the probable pattern people m a given occupation would 
obtain. 

13 Lack of ability and absence of measured interests in an 
occupation almost always lead counselors to disapprove of a 
student’s choice of that occupation 

14. If a student’s aptitudes are too low, a counselor will 
disapprove of hu vocational choice even if he hu measured 
interests in that occupation 

15. Relatively little attention was paid to personality tut 
scores in judging the appropriateness of vocational choicea. 
Much attention was given to testa of ability, information, inter- 
ests and high-echoed achievement. 

16. Intereit tests are used in determining occupational 
areas; ability and information testa and high-school achieve- 
ment are used in determining levels of training. 

17. Counselors make decisions regarding the appropriate- 
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ness of vocational choices based upon generalisations which are 
often without support. 

18. Sometimes when counselors consider the same evidence 
in making judgments on the appropriateness of vocational 
choices, they will not reach the same conelunon. They some- 
time* place different weights upon teats at different times, 
interpret them differently, and disagree with each other regard- 
ing these interpretations. 

Evaluation of these results might cause one to question if 
counseling baaed upon such judgments can be more than syste- 
matic guess-work. In light of the statistical results, however, 
it must bo concluded that regardless of tho discrepancies m 
interpretations and regardless of some of the peculiar logic Used 
by the counselors, the final outcomes at which they arrive do 
agree. We must remember that m the above cases the eoun- 
aelora were forced to make a decision. In many actual inter- 
views, the counselor can suggest try-outs of various kinds that 
1 provide additional information in terms of actual success or 
failure before a decision ha* to be made. Often the counselor 
cannot make a decision and docs not have to put himself on a 
limb as he did here. The student is also an important factor in 
arriving at a final decision m the actual counseling situation, 
The responsibility for making the decision n eventually his, 
and counseling which is more "non-directive'’ in character may 
not force the counselor to make a decision at all As long a* 
the counselor is to present test data to the student, however, 
he will most likely have to interpret it as either supporting or 
not supporting the student’s choice or else interpret it as being 
completely irrelevant, 
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THE DEFINITION OF SOCIAL EFFECTIVENESS AND 
LEADERSHIP THROUGH MEASUREMENT* 

HENRY C. LINE 


The Psychological Orjxmboa 

Enough experimental work has now been done to permit 
a definition of social effectiveness and leadership, and to justify 
such conclusions aa the following (1) Social effectiveness cm. 
rises of certain habits and skills which can be acquired by 
practice, (2) Leadership u an aspect of social effectiveness, 
leaders are not merely bom, they are persona who have <fc. 
valoped social effectiveness to an unusual degree, (3) Thu 
"only a few are bom to bo leaden, the great majority mint be 
followers,” is a fallacy — most people can develop and use skifc 
of leadership if they will The basis for these statement* and, , 
a set of guides for the development of leadership and lodil? 
effectiveness will be presented here. 

Measurement 

Tho scientific definition and explanation of human tram 
depends upon the development of tests which will meuun 
those traits The wide variety of measuring instruments de- 
veloped in recent yesre under the broad heading of personality 
tests ha* contributed much to our understanding of toast, 
effectiveness. These testa, or scales, or inventories, center 
around the concept of personality as the social-stimulus value, 
of a person Whereas recently there has been a trend to mab 
tho term, personality, synonymous with person or individu- 
ality, measurement has developed according to the more spe- 
cific concept of personality in terms of social effectiveness. 

* The paper was firtpirri for PSYCHOLOGY SCIENCE AND PRACTICE 
edited by El R Henry and Dougin H. Fira, publmoon «T which bf Firm W 
BlmJurt hu been delayed by tha wtr Publication oT ibn piper m with iU 
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From the many excellent personality tents, the following 
have been selected to illustrate certain positive and negative 
aspect* of social effectiveness. The Moss Social Intelligence 
T«Jl ( 1930), because it illustrates the difference between social 
knowledge sad social effectiveness; The Allport Ascendance- 
Submission Scale (1926), because it illustrate* b widespread 
belief ihat people fall into two broad classifications, those who 
dominate and those who are dominated. The Vi nel*nd Social 
Maturity Seale (Doll, 1996), because more than any other test 
or inventory it exemplifies die concept of social effectiveness as 
the acquisition of specific habits and habit patterns through an 
orderly process of learning, and the Link Personality Quotient 
Test (1935), because it if the only test the validity of which 
has yet been established in terms of leadership. 

Social Intelligence — Social Intelligence is not synonymous 
with social effectiveness An investigation of practically every 
experiment m which the insults of personality teats were com- 
pared with those of intelligence tests, tests of academic achieve- 
ment, yean of education, or records of scholastic standing, 
reveals the almost complete absence of significant condition 
between them (Link and Roslow, unpublished), that is, mea- 
sures of social behavior do not give correlations with measures 
of intelligence or academic achievement. On the other hand, 
significant inter-correlauona are found among nearly all of the 
various tests of personality, such as enravereion-lntroversion, 
ascendance-submission, sociability, sense of inferiority, emo- 
tional stability. This bread experimental result makes rnevi- 
I table the conclusion that personality tests measure ■ common • 
factor quite dutmet from that measured by all forms of intelli- 
gehce and academic achievement test* The I.Q. (intelligence 
quotient) or the A Q. (achievement quotient) measures one 
aspect of individuality, the S.Q (social quotient) or the P Q 
(personality quotient) measures a quite different aspect 
This difference between intelligence and knowledge on the 
one hand, and habits of social behavior on the other, becomes 
dramatically dear with reference to the SccwJ Intelligence Test, 

This rtst correlate* not with the personality tests but with the 
intelligence testa. This rest emphasizes an understanding or 
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knowledge of social behavior, whereas the emphasis in tho AH- \ 
port A-S Scale, tho Vineland Social Maturity Scale, the Link 
P Q. Test, is upon specific social habits. The former strews j 
theory and judgment, the latter practice. Social effeettvenm f: 
consists of a peculiar collection of habits and skills, the common ' 
denominator of which is their practical effect on other peopled; 

Ascendance and Submission . — The A-S Seale codifies a cmk^ 
copt of social effectiveness quite widely accepted both by sewn- - 
tific psychologists and the public According to this concept, ? . 
some people dominate others in their social relationships, while 1 
some are submissive, bashful, or retiring. This idea was pop* 
Jarly expressed in Scott's pioneer book cm influencing men ja \ 
f business (1911) and u also exemplified m another early books#; 
by Scrong on the psychology of selling life insurance (1922). >' 
The ascendance-submission concept pf social effectiveness, ,V> 
crystallized by the AS Scale, la corned over into the BemJ^ 
renter Personality Inventory and a number of ocher modem ) 
personality scales ‘The use of such scales implies chat a peiwn % 
must either dominate others or be dominated by them. Tbit j 
fits in with tho popular fallacy chat people naturally divide into g 
two groups. leaders and followers. To ba sure, test scores ahsw|j 
many people at the happy medium, neither ascendant nor aUb> $ 
missive. Nevertheless, tits implication is that both extremes^ 
are essential We ate here confronted by two aspects of psjp'f 
chology, so mo times called pure and applied psychology. Oftl 
Is the impersonal science of people as they are, the other if theij 
science of people as they ought to be. According to tho findjjj 
many people simply are submissive and many are daminanhtj 
and these are the cold facts The second emphasizes the stuthl^ 
of people's capabilities and tries to establish standards or ide4uj| 
for there capacities. In the light of such teats and experimental'! 
a definition of social effectiveness in terms of those who don^J 
nate and those who are dominated, those who are leaden and|| 
those who are not, if inadequate- J| 

Social Maturity . — The Vineland Social Maturity ScalgMM 
unique in the extent to which it embodies the measurement^ 
social effectiveness as a result of learning. It* construedip 
makes this clear Practically all of its 117 items refer to ir|| 
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cific habits and habit patterns graduated according to their 
complexity into 17 age periods All of these habits are of a 
bind which can be or roust be acquired by practice. The more 
of them the individual acquires, the more socially mature he 
becomes, no matter what his chronological age However, the 
grouping of habiu by ages implies that the normal person will 
acquire the habita characteristic at his age. There is no Impli- 
cation of social dominance or submission, except in so far as the 
person who has acquired these habits well may be superior to 
the person who has not learned them. 

This scale u unique also ui the extent to which it consists 
exclusively of objective habit or behavior items, at least ui 
intent Unlike so many personality tests and scales, it asks no 
questions about how the individual feels, whether he is ever 
troubled with a sense of inferiority or whether his feelings are 
easily hurt. Socisl maturity is defined entirely in terms of 
overt behavior, and not in terms of ambiguous introspective 
responses to subjective questions Through this scale Doll 
almost seems to say. "If a perron will take care of his habits, 
his feelings will take care of themselves." If this was not Doll’s 
intention, It certainly was that of the author of the P Q, Tut, 
in which the items are almost entirely habits and activities, and 
where such subjective traits 11 emotional stability, senw of 
inferiority, of security, and of social acceptance were deliber- 
ately emitted 

A person’s feelings or emotions as a result of social situations 
are unquestionably Important However, both psychological 
theoiy and the trend in the development of tests and m clinical 
work justify the conclusion that feelings of social inadequacy 
leading to a general emotional instabdity are not the cause of 
•oaal impotence. They are the result of failure to acquire 
adequate social skills Therefore, while the emotions of frus- 
tration may give us broad duet to the problems of social compe- 
tence, the achievement of social effectiveness must be defined 
ID terms of the habits and skills which make it possible. 

Doll’s Seals is not divided Into traits, as are many person- 
ality tests. Nevertheless, some of the items in his scale include 
such a complex range of habita that they might almost be said 
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well have been given some such name as Social Effectiveness 
Quotient Tut, since it u concerned entirely with habhj and 
skills in this category. These habits are classified in four groups 
in the test, representing the following four craicat (1) habits of 
social Initiative, (2) habits of self-determination, (3) habits 
of economic self-reliance, (4) habits of adjustment to the oppo- 
site aex. The title, P Q. Tut, was chosen partly to emphasize 
the fact that these habits are the result of learning, and partly 
to empbaaixe the sharp contast with the I.Q. Experiments 
with many personality tests, as well as with the P.Q Tut, have 
shown that the habits of an effective personality or a socially 
effective person were ui a category quite distinct from the 
habiu of scholastic intelligence and achievement. This fact 
was confirmed in the nation-wide experiment to validate the 
P.Q.Tut (3,8). 

Leadership is a manifestation of social effectiveneaa Thu 
waa assumed when the PQ. Tut was validated with refer* 
cnee to tho criterion, leadership. Leaders, for the purpose of 
the experiment, were high-school students who had bean elected 
by their classmates to positions of leadership, not leaders ap- 
pointed by teachers or school officials They were the class 
officers, officers of the General Organization and other school 
societies, editors and manageri of the school papers, and cap- 
tains and managers of athletic teams, school band and orchestra, 
gin dub, and other gimps. The non-leaders were those who 
held none of the positions of leadership, The leadergruup was 
found to have a significantly higher P Q. than the non-leader- 
ship group. Therefore, the P Q Tut measurej social effective- 
ness in terms of leadership. 

Leader xkip 

Thu concept of social effectiveness which we have been 
developing carries us a step beyond that of social maturity in 
terms of the status quo. It introduces a qualitative factor, 
namelyi what habits and eldHs contribute more than others 
toward social competence and leadership? Accordingly, much 
of the experimental work with the P.Q. Tut has concerned 
itself with item analyses and evaluations of the various habits 
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to represent ■ unit trait. For example, the item, "assumes 
personal responsibility," may wall include such a net-work of 
habits as would justify the trait description of seif- reliance. 
In fact, most of the traits which other tests moke explicit, plus 
a good many more, are implicit in the items of the Social 
Maturity Seals, 

There is one important exception, and that ia the collection 
of habits which have to do with adjustment to the opposite sex. 
Nowhere m thu scale u the specific and distinctive character 
of these habits recognized. On the other hand, clear recogni- 
tion u given to the importance of habits and skills which con- 
tribute to social effectiveness by enabling a person to be inde- 
pendent of people, to get along without them. In children thu 
consists oF being able to dress alone, to eat without help, to go 
places alone In adults it consists of such items as "creating 
own opportunities” and "systematizing own work.” The im- 
portance of such habits in terms of social effectiveness is so 
obvious that it u easily overlooked For instance, how effec- 
tive, socially, could a healthy adult be who had never learned 
to dress himself? On a higher level, this fact is expressed by 
Emerson's epigram* "The test of true friendship Is the ability 
to do without it” A person who is entirely dependent on hu 
friends, like an adult who hai never, learned how to get along 
without his mother, is limited in his social effectiveness. 

The person interested primarily in the practical problem 
of becoming more effective socially would do well to use the 
Social Maturity Seals as a guide, From the Standpoint of 
scientific psychology some of its details are open to criticism. 
Nevertheless, it represents one of the soundest attempts to 
measure social effectiveness thus far The next great scientific 
task is that of delineating the habita of social effectiveness with 
a curriculum and senes of texts for instruction comparable to 
those in th e academic field today Doll’s Social Maturity Seats 
poults the direction of this development. 

The P Q . — The Personality Quotient Tut was constructed 
to express the definition that* "Personality is measured by the 
extent to which the individual has acquired habits and skill* 
which interest and serve other people” (3). It might just as 
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and activities. The following guides for the development of 
leaden hip and social effectiveness have been deduced from 
these and other Investigations 

Social Contact , — Social effectiveness cannot be developed 
m a social vacuum. One learns the skill a of serving other* by 
working and playing with them, not by remaining aloof. Per- 
sons who take part in community affairs, or who have been 
setive members of a Y.M.GA. or Y.W.CA , drama rig club, the 
Scouts, or pthcr organized groups, were found to have a higher 
P.Q. chan those who had not. The frequent and often sneering 
rejection of group activities is eloquent proof that individuals 
can deliberately chooie not to develop the skills of leadership 
Parents who do not believe in letting their children do a^they 
please, but who, by example and discipline, influence them to 
join the proper groups, are at least beginning their children on 
the path toward leadership. 

Organiud Groups , — Highly organized and ‘competitive 
groups do more to develop leadership than do loosely organized 
or casual groups. The habits of teamwork and good sports- 
manship are the very foundations of social competence and 
leadership One question of the P.Q Tut is< "Which of the 
following have ycu engaged in frequently in season, — baiebgll, 
tennis, bridge, badminton, hockey (a hat of 20) f " Those who 
had engaged in 6 or 7 cm thu list usually had a higher P Q. than 
those who had participated in only 1 or 2. The next question 
is: "In which of these did you practice hard and regularly so 
that you would become good enough to enter a contest?” Such 
practice contributes even more toward a high P Q. than does 
casual participation Among young people the greatest angle 
influence in die development of leadership was intensive paff- 
tlapstioa in competitive games and sports— however, not in 
one but in several. 

Prominent in this list are orchestras, glee dubs, and choirs. 
Few activities require more intensive praetkp and team-work 
than on orchestra Not only must the individual leam to play 
his own instrument adequately, he must leam to play it in 
harmony with twenty or thirty others (hie sour note may 
destroy the entire effect The best violin becomes the first 
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■virtin, i leader, u! every eomjetent playet may become cb» 
leader Bt tuoel with e sole - 

Pnimle Ptactui— The practice oi some enenl eloUi m 
private i. euontial to Ae development ol leadetebip. The ter- 
ser who rpcnca day a/ter day praeuclng the piano m aolibide 
u developing a .kill which may tratutam aeU-diatmat mto 
eocial confidence, Almon any akin which one develop* to the 
point oi competence may give him ot bet a Benin oi cnn64Mwe 
and a degree o f leadeiahip. Nor doea it have to be a akill m 
moelc, art, or literature- The mote homely ara, inch aa eooh- 
mg, touting, and tewing, telling atonea, gardening, being handy 
with toola, if developed to the point oi allpenonty, can nil con- 
rtihute toward aocial effeotiyeneea, even to leadetahip. Nrarty 
every parson can acquire some skill to the point where he exceU 

m the ability to mtoren and mtvo when 

Economic Independence .- The doye^mrat of 
independence 1* indispensable to leadership The habits ot 
doing locially luelul work, as represented by the tnit of eco- 
nomic aelf-rthance in the P.0. Tut. arc of banc importance 
Aeido hom the erpetiment.l and clmieal eyidaeiee. lot thu ran- 
dunion, it aaema only nrtural that the perdon who leadi other, 
and upon whom others depend, ahould bt independent wvd wlf- 

Sffaf.^-SociBl eBecttveneM include! habit* of deal- 1 
m. with tbs oppoaite sex. Psychological practice show* that 
’ there are some indivlduala who live in only one half ot the aocial 
world, either the world of men or the world of women, lhey 
have developed emotional and mental baniere which keep them 
from dealing effectively with mnniben of the opptmi* «e* even 
in die moat aimple social situations. Such a barrier u lUwly 
to ni» whenever an individual baa faded to develop, through 
practice, some of the elementary habits of playing or working 
with member* of the opposite sex. Many of thwe habit*, like 
aocial dancing, are obviously of a very specific tort. However, 
even walking with, talking with, and telephoning to member* 
of the oppoaite sex am found to be specific habit* which must 
bo and can be cultivatad by themselves Specific habit* of 
interesting and serving members of the oppoaite sex are sharply 
defined element! m *«W effectiveness 


Bodily Movement — Bodily movement is the common fo. 
nominator of the .activities which develop social effeenvenau, 

If one examines the activities mentioned above, or the items 
which make up the Social Maturity Scale and P Q, Tut, it ml 
be seen that all of them, whether it bo playing tha piano alone, 
playing in an orchestra, working for pay, or playing in a bue- 
ball game, involves physical movement The veiy word, 
leads r, implies movement and action The leader u one ^ 
moves in advance of his follower*, one who must make unuiuil 
exertions in their behalf In terms of our definition of social 
effectiveness, he has won his position of leadership by his ability 
to interest and serve others. In so far as he has become the 
servant of all he may become the master of all. 

The common denominator of those purauita which hinder 
the development of Leadership is the absence of bodily move- 
ment, the absence of creative exertion. In this classification 
faB those time-consuming pursuits in which the individual j 
manly aita and takes thing* in— -the movies, the radio, casual t 
reading, and just plain sitting. j 

Lead-led.— Leader* must leam to be good follower* most .1 
of the time. The condition of becoming a leader » the acqui- 1 4 
lition of skills which make one a good follower, a good team- j 
mate, For every situation In which a person acquire* these | 
skills to the point of leadership, there are many more situitioni 1 
in which his social effectiveness depends on the skills of bemg j 
a good follower A study of leaders and non-leadere (10) u 4 
which 40 outstanding leaden and 40 non-leaders were selected j! 
ui each of three high schools showed that the leaden pjrtia- ' 
pated in an average ot 6 6 activities compared with 1.7 activi- ' 
ties for the non-leaden The most frequent extra-curricuJar * 
activities among leaden were competitive athleticB Moreover, , 
the leaden had over twice as much spare time as did the non- 
leadera, in spite or the fact that they engaged in four time* is 
many extra-curricular activities The authon attributed this 
to the fact that non-leaders frittered away much of their time 
uid spent more time in reading and solitary activities whereas 
the leaders used all their time in a better organised and more 
efficient fashion. The participation in highly organized nctW- 
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ties had brought about * greater integration irrtheir lives gen- 
erally. “The fact that the loader* participate in game* and in 
attending partus,* say the authors at one point, “is highly sig- 
nificant In games the leader guns more expensneo ^witn 
people, and this experience is useful to him in leadership ” 
Especially Interesting u the light can by this study on the 
ofr-mads remark* "W® cannot all be leaden, some of us must 
be follower* * This study showed that the leader* wore also 
tha good followers, where** the non-Ieadert were the poor fol- 
lowers. That is, a Met ought be captain or top mao m <m« 
activity or two, but was merely a member of the team or squad 
in his other 5 or 6 activities In other words, the leaders were 
really Mowon in 5 or 6 activities, whereas the non-leaders were 
followers in gidy 1 or 2, or none. Thus, conspicuous leadership 
m one activity was based un having become a well-disciplined 
follower in several activities. The individuals chosen by their 
fellows to be leader* were tolerated u leader* became they were 
good follower*. 

Ctiumjhip 

The critical problem* of cituwuhip so sharply illustrated by 
current perplexities over juvenile delinquency give the above 
findings unusual importance In so Tar a* good citizenship can 
be developed through education, there finding* demonstrate the 
necessity of m almost revolutionary change in the present edu- 
cational system. This change may bo described briefly by say- 
ing that many of the compulsory element! of education should 
be made optional and many of the optional element* mado com- 
pulsory, that is, the extra-curricular organ bed group activities 
should be made part of the regular eumculum for which credit 
Is given, and many of die present required subject* made extra - 
eutricubr. 

Intelligence and knowledge are nettsuiy hut, as we have 
ifeen, *ro by no mean* sufficient In f*ct,thera is no correlation 
i 1 today between sqclsl intelligence and t°eul effeedverws* 

I JEffactlw dtiaenship depends on habit* of action m organmod 

groups, where twKt indtvldnils dtt good follower* most of ihe 
dm*' hut risp good leader* some of the time* The individual* 
* wbdiram ihe habit* of good citnenshtp in the smaller unit* 
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such a* those enumerated under tha heading, organized groups, 
thereby acquire tha basic skills required for effective parrinpa- 
tion in the larger units of a democracy. 
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A REPORT ON SCHOLARSHIP EXAMINATIONS 
GIVEN IN LATIN AMERICAN COUNTRIES 
FOR THE SELECTION OF STUDENTS TO 
BE TRAINED IN METEOROLOGY 

ALBERT V CARLIN 
UrIimI Ban Waib** Bum 
tad 

FREDERIC M. LORD 
United Sat** CM 8anrt CommkiiM 

In August, 1942, work <m begun on the planning of an 
Institute of Meteorology in Colombia under the joint auspice* 
of the United States Weather Bureau, the Office of Inter-Ameri- 
can Affair*. the Defense Supplies Corporation, and the Division 
of Cultural Relations of the State Department. 

The Institute was to award scholarship* to 200 Latin 
American young men for training in Weather Observations, 
primarily, preparing them for positions in the Tnercorologici] 
services of their own countries or of Latin American Airlines. 
Courses in meteorology, and refresher courses in physics, 
mathematics, and English were also to be presented The 
Institute waa id be staffed partly by U. S. Weather Bureau 
personnel and partly by Latin American instructors 

The long-range objectives of the Institute of Meteorology 
in Colombia were threefold. 

*1 To increase the aafety and regularity of air transpor- 
tation. 

"2, To pave the way for improvements in agriculture 
through better crop planning and forecasting. 

"3. To provide greater facilities for flood forecasting, pub- 
lic health work, and Improved operation of public 
utilities through advance knowledge of weather ” 

In the planning of the program the question inevitably aiwe 
as to how young La tin- American men were to be selected for 


the training in the Institute The men selected had to meet 
the following requirements 

"l. They must have a knowledge of physics and mathe- 
matics, or they must at least show an aptitude for 
the study of physical science. 

“2 They must have a knowledge of English, or at least 
have an aptitude for learning languages " 

The requirement of a knowledge of English arose from a 
further aspect of the planning of the Institute. After the aix- 
month course or the Institute was completed, 40 of the better 
qualified students were to be brought to the United States and 
provided with advanced meteorological training at one of our 
universities, Thus, though it was not necessary to select only 
those who had a good knowledge of English, it was desirable to 
have as many so qualified as possible, from whom 40 might 
later be chosen for advanced training in the United States. 

The young men were to be selected from each of the Latin 
American countries, the number from each country being pro- 
rated according to population, area, and number of airline- 
miles 

The first method of selection that presented itself was to 
base selection on achievement in physics, mathematics, and 
English, as shown by school records It was immediately ap- 
parent, however, that the scholastic standards of each countiy 
are different, and, furthermore, that an evaluation of the 
courses of study even within each country would be extremely 
difficult *' 

It was soon decided that m order to secure objectivity in 
the selection process, it would be necessary to give the candi- 
dates an examination which would measure such factor* as 
knowledge or elementary physics, knowledge of English, and 
aptitude in physical science and language The United States 
Crvd Service Commission was asked to design such a battery 
of tests, to be translated into Spanish, Portuguese, and French 

The examination was scheduled December 15 and 16, 1942, 
in the capital cities of the various Latin American nations. It 
was open to individuals between 20 and 30 year* oF age "who 
have completed the usual academic preparation for entrance 
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to the national university of their own Or a neighboring 
country. 1 * 

The written examination consisted of 4 objective tests com- 
posed of multiple choice items Test 1 was a mathematics and 
physics test composed of eighty items, ai follows: 35 mathe- 
matical problems in physids, 25 physics information items, and 
20 mathematics items. The 20 mathematics items consisted 
of 9 geometry, 5 algebra* 2 tngonometiy, 2 analytic geometry, 
and 2 calculus items. Test 2 was an English teat composed of 
100 items, as follows; thirty-five vocabulary Items, fifteen 
grammar items, and fifty reading-comp rthenaton items. Test 
3, a language aptitude test, was composed of 150 items, as fol- 
lows 1 forty items requiring the competitor co form words from 
jumbled letters, eighty items requiring the deciphering of coded 
words, and thirty items requiring the translation of an artificial 
language. Test 4, a physical-sciences aptitude test* consisted 
of 100 items, as follows. 2S spatial-ability items, 25 items re- 
quiring the competitor* to predict the motion of various illus- 
trated mechanical devices, 25 itema requiring the insertion of 
a missing number in a number series, and 25 items requiring the 
competitors to discover the differences m a set of similar geo- 
metrical diagrams. 

The examination was printed in Portuguese for admmistra- 
, turn in Brazil, m French, for administration in Haiti, and in 
Spanish, for administration in the remaining Latin American 
nations. The examinannn icons of 708 competitors were avail- 
able for statistical analysis The score on a single test was 
obtained by dividing the number of correct answer* by the 
Dumber of items in the ten and multiplying by 100, the total 
score was obtained by adding together the number of correct 
answers on the Tour teats, dividing by the total number of 
items, and multiplying by 100. 

Fwdtagr 

Table 1 shows the number of cater, the meant, and the 
standard deviations of total score* for the three translations 
of die examination. It is to be noted that a companion of the 
mean scores of the Haitian competitors, the Brazilian coroptri- 


TABLE 1 

ilMW oodSUndard DicUUons a) Total Scow eft (L tin, Trandodooi « I 

l£j AliWUllM 



Spaaah 

Porotjuw* 

French 


. . 553 

126 



. 49.5 

SIS 


Sandird fariition . 

. . 142 

US 

1L6 


ton, and the competitors from the Spanish-speaking nations 
provides no basis for concluding that the three translations of 
the aamination differ sharply in difficulty from each other. 

Tabic 2 summarizes the obtained data relevant to the effec- 
tiveness of the 4 teats as selective devices. The sample of 185 
cases upon which the calculations were based was obtained by 
selecting every third paper from the paper* of the competitor* 
who took the Spanish translation of the examination 

The figures of greatest interest in Table 2 are the estimated 
reliabilities of the four tests The high estimated reliability of 
the English Test can be quite plausibly explained on the reason- 
able assumption that the range in the competitor** knowledge 
of English was very great, as indicated hy the fact that scores 
on the test ranged from 0 to 96 

The extremely high estimated reliability of the Language 
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SO 

30 

12 

M 

100 

SD 

J9 

S4 

ISO 

64 

24 

SB 

100 

SB 

12 

44 
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40 
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Aptitude Tut cannot be attributed primarily to the length of 
tha tm, since reduction of the number of items from 150 to 100 
would bo expected to reduce the reliability of the teat only by 
about 0.01, nor is it plausible to assume that the range of the 
language aptlmde of the competitors was unusually peat, in 
spite of the fact that Bcores on the test ranged from 2 to 95, 
since presumably the range m language aptitude among the 
competitors u roughly the same as would be found among a 
group of high-school graduates in the United Sutra One 
plausible explanation that suggests itself is that many of the 
competitors may have completely failed to understand the 
directions of one or more of the 4 different types of items con- 
tained ,n the test K this actually occurred to a large extent, 
it would be expected that the test would have high raliabihcp 
as a result of tho fact that the test would function as a highly 
reliable measure of the competitors' abihty to understand the 
particular directions contained in the test. Such a test, how- 
ever, would he undrairahle for tho present purpose since not 
only would the test be a leas effective measure of language apti- 
tude than it should he, but also .t would not even be a good 
measure of ability to understand directions other than that 
particular not of directions contained in the test. 

The estimated reliabilities of the Mathematics and Physics 
Test and of the Physical-Science Aptitude Test are not so high 
as would be desirable In the cam of both there tests the low 
estimated reliability ia presumably attributable to the fact that 
both tests appear to have been very difficult for the competi- 
tor,. The ranges of scores on the two tests are from 0 to 70 
and from 6 to 74, respectively. For the present purposes, the 
low reliability* of these testa are not so serious as might at first 
appear, since recent developments in test theory indicate that 
m the case of a very difficult test the scores of the higher-scoring 
competitors are probably more reliable chan would be indicated 
by tho reliability coefficient calculated from the Mores of the 
enure group of competitors 

The intercorrela lions of the tots are not so high, as to indi- 
cate a serious degree of duplication among the testa As might 
have been anticipated, the correlations of the English Test with 


THE 1943-1944 WORK-SESSION OF THE COUNCIL OF 
GUIDANCE AND PERSONNEL ASSOCIATIONS 

T«B second Work-Session of the Council of Guidance and 
Personnel Associations was held at the Hotel Bdtmore, New 
York, on Thursday, November eighteenth The meeting* 
were scheduled for November instead of January with the 
thought that the representatives of the constituent organas- 
pons might profit from an earlier discussion of current problems. 

The morning session was largely devoted to the general 
subject of Rehabilitation. 

"Summary of Findings of 1942 Meeting with Emphasis on 
Changes during 1943" 

Lieutenant C. Gilbert Wrenn, Bureau of Naval Per- 
sonnel 

“Rehabilitation of Service Men and Women® 

Mr. William Gilchrist, Chief of Rehabilitation*, Vet- 
erans Bureau 

Dr Harry A, Jager, Chief, Occupational Information 
and Guidance Service, United States Office of 
Education 

“Rehabilitation of Working Students" 

Miss Beatrice McConnell, Director, Industrial Divi- 
sion, United States Department of Labor, Chil- 
dren's Bureau 

Gilbert Wrenn gave a brief hut very complete summsiy of the 
work-session of last January. He pointed our that we are now 
living out the plans discussed at the last meeting; and we are 
now anticipating the post-war problems. Mr Gilchrist re- 
viewed government procedure as n is at present in Operation. 

Dr. Jager spoke for the educational program that is just now 
getting under way. Miss McConnell reported briefly on the 
dislocation p( the school population and the Inevitable ctuire- p 
quences which will some day have to be dealt with 
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the other three testa am the lowest of the obtained condahons, ! 
averaging roughly 0J5. The mtereorrelanone of the other 
three tuts are all close to 030, indicating an appreciable, but 
not too serious, degree of overlapping 


-Although Latin Americans have limited experience with 
objective tests, since moat of their school examinations are of 
the essay type, the Latin American officials and the competi- 
tors were in general highly pleased that the award of the 
scholarships was made on the basip of merit determined by 
objective tuts, rather than by personal selection On the whole 
the project append to have been highly satisfactory [iom the 
point of view of all concerned 
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The afternoon session emphasized the need for the “Freseiv- 
virion of Social Values in a Time of War " The speaken wpja 
Mr Robert P. Lane, Executive Director, Welfare Council o (> 
New York, and Dr Harry D. Gideons*, President, Brooklyn 
College Mr. Lane's speech was a philosophic approach to the, 
social values in our civilization. President Gideonse discoursed 
on the social values which our educational system has tended to 
emphasize 

At the dinner meeting tbs speaker was Dr Luther GuJidf, 
Chief of the Planning Division, Office of Foreign Relief and' 
Rehabilitation Dr. Gulick had Just come from the meeting! 1 
in Atlantic City and apoke on "Rehabilitation Abroad." - 
On Friday the Executive Council of the American College 
Personnel Association held its annual meeting. The members ' 
present were E. G Williamson, President; Gilbert Wrenn, Vice- 
President, D D Feder, Secretary, Thelma Mills and Helen! 
Voorftera Other membere in attendance were A. J Brum- , 
bangh, John G Diriey, Winifred Hausam, Forrest Kirkpatrick, t 
James Mcdintock and Robert Moore. * 

The necessary business was transacted and then the groups 
discussed at length some of the problems of post-war cuunstJ-f 
ling. The results of our deliberation will be reported to you is^ 
due time. J 

Helen VombseS ^ 
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1916 ISn/od-fftort and rtttaied on ilieJW Arnlred SMn/o^Jto« (Fmn L) 
tv, -mb duMobeeil era at iha rime of aha fiiat tot wia 5-86 yaara and at are 
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GUkioaoi, Howird. “Tha Saaihore Meanuaa of Muaret Talent red Speech SUL” 
tonreaf a! Aftphed Ptyehobay, XXVII (194J), 441-448. . , 
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(I) nuonal laboma, throe of tha woman wren (II faetoay wot beta it land 


1 , 0)0 


n (If way wwbere at land orera- 

— r““— , — «■ *. (3) pnekrea and wnppm, (4) aahi data, 

-joonc wo ike n The ireulu daw that dfmani do appear 
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Vtfl (1943), 161-164- . . 

A renenl fomda tor dm relkhliiy d a weaghcad cmnppwu bar jwen dren^ 
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T«*i 1 dmfemmw to dirtoi Sirutturrr . by Col H, If Drake, Entl E. V 
hetman, Alaini In Joseph, Louis Teicfamin, ind Uit A ranoo- 

Tat 2 e A SuT >wr* by 0£ R. H. Drake, Em K. V J«Wo, J»™s L, G 
Flaps me k, sod &H A. Prime Time, 40 rnhuilee. 

Tat li Mritoreiaw, by Col R H Drake, Em, K. V Jicinun, end Rosalie K- 
gmcboi. Tnc. 40 mum it*. 

Tot 4i No tntrnfoa, b^CoLB. H. Duke, Ene. K. V Jicirnsn, end Men Rnth- 

Tot Sj Pan /, Radio to P*i II, CM Air XmCtU*. by CcL. 

R. H Drake, En*. TL V Jackman, and Alexander Joseph. Tint, 40 

Thae teenTanftof the llih and 12 th pi da ind Juilor coUmlmL Norm tor 
ub] to part atom Tor ike Ilih mdi ate fonhhed. Each booklet, 10 re 99 
copra, Ml 100 [or m«e wto, M# Speonwi at, bidudmi tree and e»4 
buUtLSM. The tnt» niy be hind- or pta nun Mowed miclune-ecs red 1 amt 
.beeS/li* each. Miehina-Koriaa nod. 10*. PuWUrel b T die Cooperative 
Tat Seirkn, 11 AmiUnUi* Amu, New York, N Y 

Coopmlb* CcUut B*tory Tin, PncUonJ Torn T, by Ceil A- Pamm and 
ouhjs, WOTore, W lauuta. Tha tret - for im-ynj codt«bi^of 7 dura, 
(or which tentative oertentaa norma an provided. 10 to 99 euplre, 7* each I 100 
a pore tophi, 61* Specimen act, Ths tat majr be bind, or muW 
•cured marfune-Kond * newer tfaami, 2* each miehbc-ecomif au«D. 3M, 
j^iNahed by «b* Cocpeniivo Tar Sam, II Aaeteidem Annus, New York, 

CoafTMth* CaSUt* UsHrmUkt T«l, ProoWoael ha T, fey Emma Bpaney and 
TFmau dope, Maijuoits D DsHrow, Richard L H«y C, V Nam, Fred 
Robertson, udE R South, 194] Tone. 40 murala Thh tat m for fini-yesr 
Cullen n m bsiasrire ditto. 10 in 99 wpire, M, IDO or nan moire, 51* 8p«±- 
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shears, li* eich, nuchine-wumr Mend, IS* Published by the Gnopendve Text 
Service, IS Amsterdam Avenuj, N«w York, N Y 
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SH D n ■* H A, Graane, A N JorKenaen.and V H idklM? 

TTma, *5 ainnM Iju ten la lor hfgh-Khoolatd nhep tmdenilor whkA 
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■Rawer dicta, J 2.00 per tntxuand. For ule by Superintoidait of Dsmim* ' 
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PrtnnodTort, by KF WonJerilc. Foram A and B For adnln. Time, U min 
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In Jidy a m* Knhouw Glbt a s ch ool eras opened la Qikapi. Dorothy Coaes, 
who had Indybeo Pis cement Dntenr In the Breton school, was reads Dinctnr 
of the Cbictft ached 

Non A. Qmtdnn, Qdondo State CoHap nf EJuaikm, has bean to Waahiajpoa 
on a tanponnr nre mi n n H with (he Coil Aqpnaaacl Adstinntntm n fcdunnoaal 
CorealtaM, Avurioa Bdacanan Saxvfcw fta CAJL b H|ndaa with ihn 
Antenati Crana) o 


Lc. Catood Ink W Mi b Qxomindmi OEctr of SCD No. 5410 Star, Basic 
Trauma Caoiw, A5TP, Fore Brenmj, Georpa. Tbw uBN * lha dauiSauoo. teat- 
toy, tad itanmol enur for procowny Aran' Ground Forcre Panoond Qdood 
Craft uyt tly work rewlaub to way ntucb j/ r ba taofk of du sdmlato offa ood 

■ Bia formed for IfKillU 
irjt aanboi of andntota 


7 — 4 and cnnulal n 

wort ■> that they can ctMptady datny and k 
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Win. C JotoMk Jr-, Dktetnr qf Ftoremt, 1 Vk«inb Polytechnic Tmthuts, has 
tcecptad a poattxm wn da Goofyssr Alrcrslt Corpora uoa m Akruo, Ohio. 

Harry W Smmaaa, Gann) Secretary of toe fm Sou Qmdu A 
Myltana Snte CWU»e, b . - - - 


hmwtm Adviser, Nadaul Hotmt Afeacy, QBca of lha Adrembtnrer, Waah- 


■ of Edna bom! Pay- 


Sore early « 1941, ftwoU E. Super, Andatc Preftm , 

itothw and Director of the Ptncond Hmaaq at Clark Unhrwuty. haa baa an leave 
of akaren fw dmy wnh the armed wryjaa. At proem he ■ a Captain in the 

ISTi X •& ssw* 

(or dw uaithm doom mdooe mb analyib, and drvilnrimmi of tats tot pflou, tuvi- 
■am, Md bnnbanbtfa, and hare omfved such ihuus as field - 

rnmian are m a , ind ym pin 
Ha ha a nmahar of twin red p 
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““ — J « pirdiBy threat p0o< niiA| si a psrucipaot abere*er. 

■red ptychotofma ** ■nnwni» irt the Tret DcretopmeM 
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. Lbnt (I*) E L, Swarm, UAN R., 1 
of IVreoond at Ouw Mountain Junior 


Sal enire 
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mttd a leave of abtawc a 
„ . - , A B “to die Navy as 

AJ«r reman wm nuunha U Paychdoam at tba U S Nava. 

. n Board, Dallii, Tetaa, ha was order ad re tho II fi Naval Kliaht 
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J E Wdteri, Vice President In ebarge of Peraonnei and Labor , 

Sots: “* c -’ — ~ 

Public inj uninran rat haa rectady been made by tba War Manpower 
•an a inacd policies and proccdiirre 01 cunoretiKi wish dn placemnu ai 
larenre of Mnorwwl, indudim prafarefonal to toennfic pmoaatl name 
5 * .l !!?*“ ^r“ Mf Tfe** “ w tcyulltiain arc embodiad in Reuilitim 
115 ll?B to B KT W Cammm,aa Sda&v* Servian Local Surd Me 
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AN INVENTORY OF STUDENTS’ GENERAL 
GOALS IN LIFE 


HAROLD B DUNKEL* 

Cm pm ' pro Study k Gian) Educilnn 


What basic beliefs notv constitute college studentt' "philos- 
ophies of life” or "designs for living"? Whac does the College 
student consider the main goals of his life? Far the sake of 
what values docs he think that he lives? To what extent does 
his total pattern of values seem to meet certain criteria of a 
"good" design for living? The attempt to answer these and 
similar questions led to the project in phriosophy of life and 
religion undertaken by the Cooperative Study in General Edu- 
cation 1 Almost without exception the colleges stated some 
such institutional objective as “to aid the student in developing 
a useful and deniable philosophy of hre," and faculty members 
insisted that their institutions did not merely stare this ob- 
jective in the catalogues but were sincerely striving by various 
means to aid the nudent in developing an adequate personal 
philosophy 

Four passible steps in dealing with the problem of students’ 
philosophies of life can be indicated by the following questions 
I Do and should students hate philosophies of life? II, If so. 


1 Work on dill project hi* been niticd on hr faculty number* of collegia par- 
tmntMf in the Cnopmliaa Srudy la Oaull Education and by mccntierm ci the 
central ant wbef ui the SeU of Hiranttiliea, Gtor*e E. Barron, jr , Walker IL 
HB, and the author Dr Bamm, now Lt Barton of tbs U S. Air Cnoa, direct* d 
the original mpmubun of liic project and tfw preomnoo of iht fint form of tho 
Inventory Thi coin pitta report on I ha project, including norma and din on ft) la- 
bility and validity, a beam puhl*H*d tkn aoaimrr 

■ Tha Cooperative Study k an unninuoD n appmumaidy ■ cam of collet® 
who afnek ctwpcmivdy, with (hr aid of a central a tiff, thru common ctlocatwail 
proMama. For a «pt rt of the finidy't work, ice Ralph W 0*1 n, The Coo pm the 
Bhjdy w Garni Ed .cum," Tk, Bfa* ri«wf jBBfl [Ocrohtr, 19*2), 

ktartMod m neater detail about the development of thu projert, 


- ------- m area ter detail about the development of thu project, 

5K7Vr7"fe 4" ^ rw> '^ Urr Ohs Cooper* ova Study in General Edoauw. 

SiJS Kim but Ave, Qucips, mioata), vet II, do 6 


what are these philosophies? Ill Once we have ascertained a 
student's philosophy, do we (the student and the faculty) con- 
sider it satisfactory, and on the basis of what criteria do we 
decide? IV If a student's philosophy appears unsatisfactory, 
how can the college assist the student in working out a satis- 
factory view of life? 

Our colleges believed thzc the answer to the first question 
was "Yes," that students should have a philosophy. Many 
teachers and institutions had made a practice of having stu- 
dents wnte essays setting forth their philosophies Since these 
essays had all the limitations of any verbal statement (they 
are what the student thinks, or wishes to think, or wishes the 
reader to think, is his philosophy), these teachers were of 
course not under the misapprehension that there is a perfect 
correspondence between the way men live, and the way men 
say they would like to live The hypocrite, the person with 
selfish or antt-social aims which he fean to express, the victim 
of social or economic pressure, alL undeniably exist Those 
undertaking the project believed, however, that it is equally 
incorrect to assert that there is absolutely no relation between 
the way in which people talk about life and the way In which 
they live. The group felt that in many cases the verbal state- 
ment about the kind of life the student was seeking would be a 
useful index, even though not a perfect one, of the kind of life 
tho student Was living or seeking to live. 

In the case of many students, when there is s discrepancy 
between stated beliefs and conduct, this inconsistency exists 
without the person’s being aware of il For these students, an 
opportunity to state their beliefs precisely and to examine the 
implications of those beliefs for action may often lead the 
student to harmonize more closely his pattern of beliefs and 
Iiis pattern of living. 

Other students actually do not know what they believe. 
They have never thought much about their way of life or about 
what they believe- Having acquired their schemes of value 
rather unconsciously and at random from various sources, these 
students, when they leave the pattern of life into which they 
were born and grew up, and when on coming to college they 
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meet new and perhaps conflicting values represented by new 
environment, new friends, and new experiences, have consider- 
able difficulty in ordering Aeir personal lives for the first time. 

Without help in seeing and resolving some of this conflict 
and confusion, the student may go through much if not all of 
his adult life, a victim rather than a master of these warring 
values The purpose of the proposed device was, therefore, to 
enable the student to see clearly what he says he believes, to 
compare it with the philosophies of other students, and to ob- 
tain help in overcoming difficulties and conflicts in his own 
position. 

Tho group did not believe that all these results could be 
satisfactorily produced at the- verbal level alone. Even on the 
verbal level, we certainly did not wish that a student should 
work out for himself, once and for all, an immutable philosophy 
of life We believed, however that 


most mdien , mil agree that no student should merely drift 
ttapuph life, allowing his major decisions and actions to be derermioed 
for him entirely by arcumstances — to be merely the resultant of 
external fortes impinging upon him Each student should bring to 
erery important ufo decision some "sense of values” which is hu own 
. » Of course sore a "design for living" can never be completely 
npueit . . In a brief, oversimplified, present a non It [the idea 
of a design J sounds unduly rationalistic — as if each student were ex- 
pcctcd to chart all the details of hu life, to be corua our of all tho 
*%?'**'* act j “d tD "fcraH decisions to some grandiose 
nodon of life and the universe We do not mean this » 


Rather tho philosophy jhould be n "working hypothesis 11 for 
living, to be revised as and if necessary, ui the light of new situ- 
ations and insights Though a satisfactory verbal statement 
of a philosophy or life is not the whole journey, it is at least a 
step in the right direction. Such reasoning as this, then, led 
our group to the belief that students should have a philosophy 
of life (the group's answer to the question of Step I) and to an 
interest in securing a verbal statement of this philosophy (Step 

A further^ result of conferences between faculties and stall 
^ that iomo objective means of securing and re- 

*** Hmslttur, VbL II, do. 6, p 6, 


curding the student’s statement of hu philosophy was desirable. 
Although, to be sure, tho response m tho form of an essay gives 
certain insights which probably cannot ba made available 
through objective devices, the existence of the objective in- 
strument would not preclude the use of the Baggy 

On the other hand, the objective instrument Would have, 
the group /e|r, several advantages In the first place, the 
philosophies" thus recorded would be more exactly and more 
easily comparable. For example, the score of one student could 
then be compared to his own sporep at other points in hu edu- 
cational career, to the scores of other individuals, and to the 
scores of groups. Similarly, scores or sections, classes, or entire 
student- bodies could be compared, and if desired, judged. 
Likewise the phdosophy of a student who, when left to himself, 
deals in any generalities would have more points of comparison 
with that of a student who plunges into petty mlnutlaej and 
the statement of the verbally facile student could more mean- 
ingfully be compared with that of one less articulate. 

Furthermore, the records of these philosophies would be 
more concise and more usable. Faculty-members who wished 
to learn about the philosophy of a group of students would no 
lpnger face the difficult and often impossible task of reading 
through hundreds of pages of student manuscript. Rather, 
the results stated numerically would permit fairly rapid sum- 
marization end comparison. Then finally, instead of existing 
like the esgay in a single copy tucked away in some pereonnel- 
folder or teacher's files, copies of die results of an objective 
record could be made available for study and consultation, 
easily and inexpensively, to every person concerned with the 
student For these reasons, then, the preparation of some ob- 
jective device of this sort seemed eminently dsmble as a fint 
step 

Many aspects of the student’s philosophy of life deserved 
study; but, for various reasons which need not be discussed 
here the group began work by studying the main goals which 
students hold. Hence, though other studies were planned to 
supplement the information gained^ from this device, the fint- 
lnstrument which the group undertook to construct was the 
Inventory of General Goals of Life." 
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First, a list of possible main goals was secured from papers 
written by students and from teachers in the group. In the 
selection of the goals finally included in the uiventoiy, two 
criteria were dominant 'Hie first waa that the goals luted, 
though couched in student phraseology, should give an ade- 
quate representation oF certain of the great historical traditions 
of philosophy and religion The second was that the list should 
also include goals which, though less common in formal philos- 
ophy and religion, are familiar in our culture or in "cracker- 
barrel philosophy" Within the limitations imposed by prac- 
tical conditions, we wished each student to find expressions 
which he could cdnndcr adequate statements of elements of Ins 
own point of view. 

The number of possible "philosophic" positions (to use the 
term m Its widest possible sense) and the number of statements 
that can be framed to express any single one arc, or course, 
enormous. Yet the practical situation demanded that the final 
lilt be extremely brief, We tried to secure brevity in three 
ways. (I) We allowed certain goal* between which distinc- 
tions are customarily made in philosophy and which yet appear 
to be closely related to one another m the minds of most stu- 
dents, to stand together m a single statement * (2) Certain 
goals such as the attainment of Nirvana, which are familiar In 
the history of thought but which are not commonly held by 
American college students, were omitted (3) Certain goals 
common in student philosophies but m a subordinate position 
were also omitted ' 

Closely connected with the selection of the goals was the 
question of what technique should be used. Since the purpose 
Was to get the student to rank the goals ui order of their im- 
portance and acceptability to him, the device of "paired com- 
parison" appeared appropriate since this technique facilitates 
accurate and easy ranking of various possibilities. Thus each 
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of the twenty goals u paired once with all the other nineteen 
and the student is asked to choose one goal of each pair Ai a 
result of this process, the goal for which the student has the 
greatest preference will be selected nineteen tunes (i cvtty 
time it appears) or will have a “score” of nineteen The stu- 
dent's second choice will be selected eighteen times (a score of 
eighteen), being rejected only in the pair where it appears with 
the most acceptable goal. The other goals follow with dimin- 
ishing scores until tlie goal which the student finds least accept- 
able or rejects most emphatically is reached In short, die 
student who manages to make his choices with perfect con- 
sistency will give to goals “scores” which will rank them m 
order,* 

In interpreting the inventory, one looks first at the goals 
which are ranked at the head of the list (The exact number 
of goals considered depends partly on the nature of the par- 
ticular goals concerned, partly on the arrangement of the scoia 
in terms of gaps or tics in the ranking.) These goals at the 
head of the list are the statements which most appeal to the 
student Next tho goals at the foot of the list are considered 
since the student's philosophy as indicated by the goals most 
readily accepted is often further defined and clarified by the 
goals which he refects (or accepts least readily) 1 

The way in which the inventory records a philosophy of life 
can probably beat be indicated by an example This student 
bifora taking the inventory, wrote a brief essay stating her 
philosophy This essay is reproduced here aqd followed by a 
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luting of the order rn which this student subsequently ranked 
the goals of the inventory. The caso is fairly typical in that, 
while the statements in the phtlocophy are not verbally identical 
with the wording of the goalt in the inventory, the general 
nature and tone of the philosophy arc reproduced 


Afy Pkilaiofky of Lift 

I find it difficult tn sure my philosophy of Ufe One** philosophy 
[• far from being an immutable thing, ana nuns is constantly chang- 
ing from contact wuh the outside world. The best I can do u to 
state the basic principles which are elementally ■table, 

I first of all think that life u to be enjoyed Happiness no matter 
how hnef, leaves a luting impression that lives on tram one period to 
the next Its memory nils the more somber moments with a hope 
for better china to come If 1 am happy I Can be successful, and 
success gives the impetus to do better c rungs. 

Every man on earth is here for some purpose, I believe that I 
can attain that purpose by dame the best I can with everything I do. 
The Hung at which I waa intended to excel will show itself ut the en- 
joyment I derive from doing u At the present time my greatest 
source of joy is in writing I cannot say yet ebat writing is my talent, 
Wist I write is immature and sometimes I become very diaamragnd 
when my words do not match my thoughts But to know that 1 
have written a line full of life and color gives me more pleasure than 
any other thing I do 

I siq nor concerned with the Ufa to corns after this ena. I do not 
pretend to know what it holds for me sr how I can change it. I *m 
concerned with the present It is of too much importance to be con- 
sidered as ■ doormat to the house of the next life. The present u a 
houseln itself and it must bo lived in. I believe (hat if this bfe were 
hot meant to be enjoyed it would not hold so many sources of joy 

It ur not my amhiuon to be remembered for generations to come 
because of some mat accomplishment If I succeed In doing one 
thing really well, I shall bo sadufiod. It waa never intended that I 
be a limwM person, but I am going to see to it that I ora a aucmaaful 


"Score" 


IxvtMory Scorej 

Go d 


18 Gettuig as many deep and laating pleasum nut of life as I can 
17 Promoting the most deep and lasting pleasure* Tor the greatcai 
number of people 

17 Saif -development — becoming a real, genuine person. 

11 4 Fine relations with other persons 

place for myself in the world, getting ahead 
the specific problems of life as they arise. 

— d, eopununeiu, stillness of spine, 
x over people and things. 
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10 Serving the community of which I am a part 
9 Self-sacrifice for the sake uf a better world, 

9 Living for the pleasure of the moment 
7 Serving God, doing God's will 
7 Achieving personal immortality in heaven. 

6 Self-dudpbnc— rfjvercammg my irrational emotions and SCO- 

■uoua desires 
6 Doing my duty. 

6 Survival, continued existence. 

4 Bang able to "take it"; brave and uncomplaining acceptance 
of what circumstances bnng 
3 Finding my place in life and accepting it, 

2 Realizing that I cannot change the bad features of the world 
and doing the best I can for myself and those dear to me. 9 
I Security— protecting my way of life against advene changes 

In her essay, this girt Ant mentions, as an aim in life, the 
desire to be happy This desire Is reflected in the goal which 
heads her liet in the inventory, '^Getting as many deep and 
lasting pleasures out of life as I can " (Of die statements jn 
the inventory this is the expression nearest to “happiness ”) 
The rank she gives to “Peace of mind” indicates the same atti- 
tude The social aspect of this same aim appears in two other 
goals she ranks high. “Promoting the most deep and lasting 
pleasures for the greatest number of people” and "Fine rela- 
tions with other persons ” 

The second aim emphasized m the essay is the desire to be 
successful, to do the best she can in everything In tho inven- 
tory this aim is indicated by the high ranking of the goals of 
“Self-development" and “Making a place for one’s self in thfe 
world ” 

In her essay, this student stmto the present and is specifi- 
cally indifferent only to the problem of “the life to come after 
this one." Hence the goal of "Handling the specific problems 
of life as they arise" ranks high ut her lUt and that of "peiaooal 
immortality” falls in the lower middle 

At the end of her list come, as might he expected in the ease 
of one who Wishes to make a place for herself in the world, the 
more passive goals of "Finding my- place in life and cheerfully 
accepting it" and merely “Being able to take it 
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Possibly these brief comments will suffice to show the more 
important relations between the essay and the inventory* 
though practice gained through interpreting a number of scores 
makes the results more meaningful to the user Some differ- 
ences between essay and inventory may also be noted For ex- 
ample, the interest in writing, stressed in the essay, does not of 
course appear in this inventoiy On the other hand, the inven- 
tory reveals the student’s point of view on a number of issues 
which she did not mention in her essay Unfortunately a sin- 
gle example cannot illustrate tho many varieties of viewpoint 
and the striking contrasts between them which can be indicated 
by the inventory, nor can it show the analyses and comparisons 
possible for the scores of groups of students 1 

It u still too early to make a final evaluation of the inven- 
tory Data are sail coming in, and studies of the validity and 
other aspects of the instrument are still being conducted by the 
staff and by the cooperating institutions, Nonetheless it is 
fair to say on the bans of the evidence now avadable that the 
inventoiy has proved extremely useful and seems a reliable in- 
strument for the purpose for which it was intended 

* Bm4k» of m««I fro*p* el inukati ini idulu an ba f trend m Suf Nrmi 
Uuir, ml IV, no. II 
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cost was enormous, but Americans have always paid hand- 
somely for their entertainment. In the field of business and 
industry, competitive advantage, often cut-throat in nature, 
took, precedence over other considerations Extended to per- 
sons, the principle found expression in hiring, firing, and fierce 
competition for jobs 

Even in education the pressures against reasonable selection 
were legion, and there had been marked lag in the adjustment 
of opportunity to individual and aocial needs For example, 
before the College Entrance Examination Board was founded 
m 1900 the colleges had been "evaluating" secondary schooling 
by means of records not even superficially comparable. When 
they had muddled through to elements of a rational policy for 
admission, some presidents still objected One, more naive or 
pragmatic than the others, "pointed out that a college might 
wish to show special favor to sons of large benefactor*, to sons 
of trustees, or to sons of public men of importance who pre- 
sumably would have difficulty in meeting the announced ad- 
missions requirements " 1 

Eventually a system of personnel selection may develop 
that can with justice be called scientific Those who have 
studied the problem know that on theoretical grounds an in- 
finite number of such systems exist as possibilities. Two 
questions would have to be answered if a particular system were 
chosen for practical application’ Which of the possible systems 
would be mut uttjvlt, and Who would select itf According to 
democratic ideology, the citizens would decide and choose. It 
would be useful to the extent that its results were sensed as- 
beneficial to the entire, society, it would be selected by the 
entire society. If the urge for scientific selection were com- 
pelling, we would now be searching for our undefined or basic 
terms; we would be exploring the postulates smallest in number 
but utefvl to attaining our aims. With these goals reached, our 
conclusions and their application would follow as inevitably as 
night follows day But generations may come and go before 
public opinion can forge e method of selection worthy to be 
called scientific, 


MAJOR STRATEGY VERSUS MINOR TACTICS IN 
MERIT ADMINISTRATION 

FRED 5 BEERS and CECIL R. BROLYER 
Social Security Board 

Civil service or ment systems are relatively new to the 
American scene of government Even the most venerable of 
them are only two generations old, but between 1935 and 1944 
the number skyrocketed until there were state-wide systems or 
ones that included a combination of departments in all but 
three states Stimulation for this phenomenal growth come m 
part from federal legislation, particularly the Social Security 
Act, which in 1939 was amended so that selection of personnel 
on a ment basis was one of the requirements for receiving grant 
in public assistance and unemployment insurance 

Passing laws is risky buatnoss — witness the prohibitioi 
amendment. Civil-sannee legislation haa sometimes been 
passed in nearly as precipitous a fashion Although the re- 
sults have been notably better, they have left little doubt that 
statutes alone, indispensable as they are, cannot make a merit 
system Practice must keep step with legal precept if public 
acceptance is to be sufficiently durable to make laws effective. 
Philosophically inclined members of the legal profession say 
that 80% of the people must be doing voluntarily what a law 
dictates if the 20% who need coercion are to be brought into 
line. 

For more than a century the American tradition of per- 
sonnel selection was opportunistic. Officeholders wore dis- 
possessed regularly with each change in administration, Hav- 
ing an average expectation of tenure amounting to two ycara, 
they spent the fiiat year learning what they were supposed to do 
and the second helping to re-elect the chief administrator 
This practice was a main source of popular merriment. The 
B7 
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Meanwhile we must consider available alternatives Among 
these the merit system seems best, although its imperfect tech- 
niques ran be much improved as this article will attempt to 
point out 

In their present stage of development merit systems cannot 
pretend to science At best they represent a compromise be- 
tween the rational and the inatinctive, drawihg as they can on 
elements of the scientific that lie snugly adjacent. Internally 
their promise of development lies in a frank recognition that 
then* is an art; that, as in clinical median^ their art u un- 
worthy of the name if any poaaible conjunction with science is 
overlooked or set aside Externally their hope lies in shaping 
public opinion by the unswerving integrity of their practice and 
the truth and cogency of their precepts The basic principles 
should be and are self-evident They should both refleet and 
illuminate tho democratic ideal. 

Five basic principles for a ment system of public adminis- 
tration may be said to have reached the stage of general accep- 
tance In a democracy they are axiomatic (1) open competi- 
tion as the method of choosing public employees, (2) selection 
by practical and scientific methods or the most competent per- 
sonnel available from open competition, (3) equal pay for equal 
work, (4) a career service conditioned upon meritorious per- 
formance of work, and (5) the right of appeal from any or all 
personnel actions 

It » unfortunate but not, 1 cm than human that disagree- 
ments over the minutiae of carrying out a principle sometimes 
become to noisily vociferous that the still, small voice of the 
principle ia lost m the general clamor. Mere contentiousness, 
however, is a minor ill. More critical distractions from the 
principles arise from legitimate differences of opinion and the 
swift crosscurrents pf deadlines to meet, techniques to refine, 
inefficiencies to scotch, rituals to perform, red tape to unwind, 
irato applicants to soothe, and special privilege seekers to 
confront. 

In the hurly-burly it is sometimes forgotten that ment sys- 
tems are never ends in themselves. They oust to serve the 
operating agencies But merit systems should be given ad- 
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mlnUtrative strength equal to chat of the operating programs 
and should have their unqualified support Lacking thii, they 
may in tune degenerate into irresponsible and malodorous 
adjuncts of the spoilt system. The relationship he tween the 
operating agency and the merit system is reciprocal As the 
one proepen, so does the other. Tf the one sinks into respect- 
able somnambulism, the other is desolate also. 

Some of the complexities involved in applying basic merit 
principles will be apparent if we examine each principle sepa- 
rately (1) The principle of open competition as a generaliza- 
tion is especially allunng in a democracy. When II L Men- 
cken mhia salad days translated the Declaration of Independ- 
ence from sonorous Johnsonese into American, he voiced the 
underlying principle in language dear to the heart — "Every- 
body is as good as svniybody else and maybe a damn sight 
better.'’ 

In greater or lesser contradiction of the principle, minimum 
qualifications of education or experience or both are often act up 
as obstacles to competition The rational Izacion is that these 
hurdles constitute protection against the cose and waste of fail- 
ing large numbers of applicants, experience having shown that 
most people with 4th grade education and wheelbarrow work- 
hutoiy flunk out on examinations for white-collar jobs Per- 
sons of great natural endowment are, of course, leas dependent 
upon formal education than others, Americans like to think, 
however, thqt Lincoln, the Wnght brothers, and Edison are 
typical rather than exceptional. So-called minimum qualifi- 
cations may sometimes be subordinated to the interests of 
special vocational or professional groups; if severe restrictions 
are imposed, they may become subversive of the principle of 
competition they were designed to facilitate. The danger is 
accentuated by the fact that amount rather than quality of 
experience is credited (Comparable data on quality of experi- 
ence are almost unattainable.) 

Local residence is almost completely indefensible as a hurdle 
to competition Yet it is widely accepted, perhaps because it 
expresses the yearning of the community to regard itself and its 
members as better then . By way of illustration it is rumored 




that every faithful citizen or Boston hopes some day to mov M 
New York and not to like kl “ 

Preference lor special groups limits the application of 0 p«i 
competition. Lady Bountiful scattering sunshine by her pre| . 
ence and gifts from her well-stocked larder is a national^ 
Except in poetry we seldom think chat “Earth gets it B price for 
what Earth gives us." For example, recent polls show exten- 
sive public support for continuing the tradition, of preference 
in the public service for veterans, their wives,' and kin. What 
would be tho offect on public psychology if proposed legislation 
read about as follows If you are not a veteran, you will have 10 
points subtracted from your competitive score on whatever 
examination you take for a position in the public service, tf you 
are a veteran but not wounded, you will have 5 points sub- 
tracted. It is not only in our efforts to befriend veterans that 
we fall into the illogical quickaand of something for nothing. 
Wo do so elsewhere. AU of us are appalled at the “mounting 
death rate owing to heart disease.'' We are brought up short, 
however, when we are asked what death rate we should like to 
see rise if that for heart disease could be reduced 

(2) Selection of the best available as a result of competition 
commends itself to the rationally minded. Most commonly 
the process consists of a written examination, a rating of educa- 
tion and experience, and an oral interview. Resulting scores 
are combined by differential weighting, each weight presumably 
determined from the relative appropriateness of the pare scores 
to the requirements of the job. At the risk of over-simplifica- 
tion, we may regard the written examination as a check on the 
powers of thinking and the extent of appropnate knowledge, 
the rating of experience, a check on stability— the staying pow- 
era of tho candidate; and the oral Interview as a check On hb 
histrionic abilities — how goad he is as an actor. 

It is in the area of the wntten examination that personnel 
administration Today cornea most closely into conjunction with 
science Full advantage should be taken, but often Is not, of 
this fact Scientific measurement has come about within the 
memory of men stdl living. Between the early attempts of 
Gallon and Cat tell to apply to human traits measures analogous 
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to those used in physical science and the appearance approxi- 
mately two decadea later of the Army Alpha lies a period of 
significant advances. Since that time, in schools, colleges, and 
in our armed forces this science has come of age, both In theory 
and in application. It has been expanded to include special 
skills and aptitudes, attitudes and interests, and is being ex- 
tended to include relatively unique differential traits In merit 
administration the advantages this science offers in " arranging 
candidate! in order* 1 u recognized 

Rating education and experience has been an armchair exer- 
cise, About all that (a reliably measured is amount of time 
spent in school or at work; the quality of the experience and itt 
effect upon the person are ignored True, those who have gone 
to school for a long time or have held responsible positions are, 
if a random group, more likely to succeed in simiiar circum- 
stances than a corresponding group of the same age who have 
not. But extensive studies by competent observers have 
shown, for example, that there is a college for every level of 
ability in almost every State, and that variability of achieve- 
ment within colleges is marked. But neither of these facts is 
considered in evaluating education Although the studies of 
work experience have not been so extensive nor perhaps so care- 
fully made, little strain is put on the imagination Sn recogniz- 
ing that evaluating work-experience and schooling are equally 
complra. What is nue of the one would more chan likely be 
true of the other. 

wo much reliance can easily he placed upon work- 
expenciiee {a made clear by three commonly practiced adminis- 
trative devices plus a factor that we shall call "survival” 
When an administrator has on his staff a difficult or inept per- 
son— but not so bad that dismissal it, under the rules of tenure, 
relatively simple — he watches his opportunity to “unload” that 
employee on someone else. The inept one moves on to his new 
position happily unaware of what has happened to him. And 
he may be similarly Unloaded several time* canying with him 
as he moves crab wise a record of lengthening and satisfactory 
j experience. If he were given an unsatisfactory record, he could 
pot be uploaded. The second administrative practice is fsrmli- 
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arly known as "lucking upataire." For equally good, or bad,' 
reasons the unwanted individual is eliminated through promo- 
tion or reassignment within the organization but at a higher 
level. The third practice is “holding down " That is, an em- 
ployee may be so valuable in a low position that the administra- 
tion cannot sffbrd to advance him In any event the record 
of the worker, 3 * seen by an objective observer, «na«ks at Inapt 
1 pare of the truth. 

As for the factor of “survival," it ia among the commonest 
of influences contributing to unreliable rating of work-experi- 
ence. In every organization people tend to get bored. Those 
most ambitious, energetic, and competent will find opportu- 
nities for advancement more readily than others. If top mao- 
agement by too htde or poor leadership contributes to this 
centrifugal force, the better people rather than the weaker ones 
will tend to find elsewhere jobs more suitable to their talents. 
These vacated jobs are likely to be filled from among those who 
are loft. Eventually persons having long; solid, C+ txpermt* 
predominate. This is the factor of survival that over a penod 
of years contributes to high ratings on experience for mediocre 
individuals and, incidentally, may staff die top jobs with in- 
competents. 

Oral interviews could be a very useful part of the ■ elec Don 
process Although they are extensively used for positions re- 
quiring contacts with other workers and with the public, their 
value it aometunea questionable. Part of the difficulty results 
from failure of eximmets to recognize the limitations pf ;he 
interview and to refine techniques that will point up ita<vlrtuea. 
Essentially the interview should be a check op the candidate's 
pwere of acting. That everyone is an actor will be imme-, 
diately apparent if we reflect For a moment on what happens 
during every conversation We keep the lid on those things 
that would give offense and try to express only those that will 
please or that will accomplish our ends with the least amount 
of friction ( When the "oral" is used as the casting director 
employs the tryout, we shall have begun to refine it for mojO 
effective use. -The easting director asks the people who con* 
before him to 'register" this or "register" that. Hu applicants. 
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who aim to please, respond accordingly; and the director then 
makes a selection from among them according to the require- 
ments of the part that ii to he played “The part” and "the 
requirement* of the part." — these are the terms that need 
definition in any development of devices to arrange competitors 
in relative order of skill 

To sum up, selection normally consists first of a written test. 
Such a teat can and should be carefully prepared since a science 
for doing this u available. It u the best method of determining 
the would-be public servant’s knowledge and his ability to use 
that knowledge The rating of training and experience is a 
second factor, contributing to the selection process Has the 
applicant been through the conventions of society, both educa- 
tional and social, that are thought to be prerequisites to com- 
petent performance on the job? Sole reliance on these con- 
ventions, prerequisites, or minimum requirements implies a 
faith in two other propositions patently false, namely, that all 
individuals have equal endowment and that all respond equally 
to experience. If these last two propositions are not true, then 
there is always a real probability that someone can bo found 
who could do a competent job although lacking minimum 
qualifications Nevertheless, the establishment of some con- 
ventions u administratively desirable and even essential But 
their promulgation by fiat should be made in a state of adf- 
awarcnesi rather than in a state or aelr-delusion These con- 
ventions simply apply two questions to the competitor la he 
reasonably well equipped for the position for which he is com- 
peting? Docs hu experience indicate dependability? Because 
of limitations in evaluation, m tbe individual, and in society, 
such ratings are purely judgmental today, that is, they are still 
unscientific Finally, the oral interview can and does con- 
tribute to the selection process. Its purpose needs clarifying 
and ita techniques refining The three in optimal combination 
yield better results than any one alone 

(3) Equal pay for equal work is a necessary complement to 
open competition and selection of the best qualified if a merit 
system is to be productive of an effective working spirit, Few 
forces are more disruptive of staff morale than inequities in pay 
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and rank. F rom the principle or just recognition for work done 
stems the durable structure of an organiza tion, tbe position 
classification and pay plan In principle equality and fairness 
are axiomatic, They need no defense. But they are as diffi- 
cult and elusive to apply as they are i elf-evident There is as 
yet no science of organization, no clear dependence of pay 
seales on ineluctable fact Even more distressing is the ap- 
parent ease with which known elements of human psychology 
and doae-to- earth experience can be ignored. 

If keen observation and fidelity to meaning of facts eon be 
called psychology, the novelists are often to be classed among 
the beat psychologists As Charlotte Bronte wrote in the 
"Editor’s Preface to the New Edition of Wuthering Heights”' 

The writer who possesses the creative gift owns something of 
which he is not always master— something that, at rimes, strangely 
wills and works for itreir He may lay down rules and dense princi- 
ples, and to rules and principles it will perhaps for yearn lie in sub- 
jection, snd then, haply without any warning of revolt, there arnica 
a arm when it will no longer consent to “harrow the valleys, or be 
bound with a band In the furrow" When, refusing absolutely co 
make ropes out or sea-sand^ any longer, it sea co work on itatue- 
hewing. . Be the work grim or glorious, dread or divine, you have 
little choice left but qumant adoption. As for you— the nominal 
artist — your share in ic has been to work passively under dictates you 
neither delivered nor could question. . , If the result be attracova, 
™ World will praise you, who little deserve praise, if It be repulsive, 
the iame World will blame you, who ilnmc as little deserve blame. 

On first reading one gets the impression that here is, per- 
haps, the best definition of genius he has seen But closer re- 
flection, stimulated by >che shift in point of view from the 
impersonal “he" to the very personal “you," suggests chat 
Charlotte's description is not of genius at all but of everyone 
who w human The ordinary or garden variety of classification 
and pay pJan under a ment system leaves much undone that 
might with a little effort mold such plans closer to the heart’s 
desire. The typical pyramidal concept leaves out of account 
much of the universal human urge for elbow room — freer play 
for talent; At the top administration, next supervision, then 
prof canons and specialized vocations, and finally the workers. 
Even at the top it is quite possible to search dozens of plana 
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and nor find one that would make provisions elastic enough to 
attract a man like William James — physician, psychologist, 
philosopher, man of letters; or Benjamin Franklin — artisan, 
scientist, inventor, diplomat, statesman 

Inelasticity, however, la not the only weakness Of greater 
jeopardy to the principle of fairness, bur perhaps more easily 
remedied, is the faQure of many operating administrators to see 
lha manifold advantages a classification and pay plan can con- 
tribute to good administration. Too often they regard the 
structure as a hampering rather than a facilitating device; by 
circumventing the requirements they put themselves in the 
position of the sick man who brushes aside his physician and 
dings to his amulet Merit systems must contribute more 
than they da now to informing and instructing the operating 
agencies — administrators and workers alike, 

(4) Fourth among the major principles of ment administra- 
tion ^ii the requirement that the system provide for a career 
service based on merit. Basic to thu principle u tenure of 
office for those who have been selected as the but available 
knd who have qualified for permanent appointment by having 
served successfully a working-test or probationary period 
Once they have pined muster it is in the interest of the public 
service that they be given reasonable assurance of holding their 
jobs. With this assurance they are free from fear, and can 
direct their energies to increased job efficiency. Bernard 
Shaw, however, cautions rhoie who start at the bottom and, 
hopefully, dimb the proverbial ladder rung by rung. You 
don’t learn even to hold your own, he says, “by standing on 
guarf but by attacking and getting well hammered youreeJf ” 
A polity of pRunotiQD-from-withui » another dement of a 
ureer service. It is earned to extrema when It is so nearly 
like the human drculatory system that elaborate preparations 
and intravenous injections are necessary for the purpose of in- 
troducing a little new blood. Very often new blood brings new 
life, as many a soldier can testify “Permanent tenure” is not 
an Unmixcd blessing It has even been called a necewary evil. 
Those who accept it as a principle of administration and those 
who profit from it as employees should understand its hazards 
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Rnd the limicationa on in effectiveness As part of a career 
service, some method of evaluating thp worth of those with the 
prospect of promotion is necessary Recourse here to the basic 
principles of selection is indicated— written promotional ex- 
aminations with a broad baae of competition, plug service or 
efficiency rating as partial determiners of “the but available." 

Service ratings leave much to be desired. They have inhw- 
ent limitations. They put a premium on conventionality and 
often on the mediocrity that never swerves from the beaten 
path They do not foster understanding of the brilliant em- 
ployee whose contribution thu week it more than noteworthy 
but whose unexpected absence the following week leaves the ad- 
ministrator gnashing hu teeth Service ratings do, however 
bring out Jnco the relative open, so that impartial observers can 
ree them, the supervisory judgments that would Otherwise be 
made behind closed doors and acted upon arbitrarily in star 
Chambers Moreover, the rating process can be refined If 
employees share in it snd understand its advantages as well as 
its limitations, it can serve an exceedingly useful though limited 
function 


— is cno ng at of appeal. 

Thu right rests on the theory that the indmdAal under merit 
administration has basic rights that cannot bp ignored and that 
rile administrator has responsibilities that he muat discharge 
Appeals usually lie when an applicant has been refused the 
privilege of standing an cumulation; or if he reels that he hu 
been unfair^ rated on one or more parts of the selection 
process, or if he challenger the fact that he was not certified 
that his name was improperly deleted from a register Ap- 
peals frequently lie for layoff, demotion, suspension, and almost 
always for summary dumual 

Many defensible differences of opinion exist over particulars 
W ix eiy ° f "PP* 1 * 1 ' actions covered, the nature and 
calibre of rile appeals body and the manner of appointment, the 
quution of informing the worker of his rights, and the question 
or the administrator's responsibility once an appeal has resulted 
in a finding But there is little if any disagreement among 
ment or cm! service systems chat the right of appeal is cardinal. 
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In this article merit systems have been criticized — not chat 
they may be destroyed but that they may be improved The 
only «Itermtive to selection by merit u selection by personal 
whim In that direction lies chaos Thus, society is left with 
the task of making its "merit system" more workable, more 
reliable, more internally consistent— in short, more nearly 
scientific. Enough experience with merit systems is now avail- 
able that some of the malfunctioning are apparent Knowl- 
edge is avadable that could be used to correct these malfunc- 
tioning^ Improvement of mem systems is contingent upon 
their scientific development, their acceptance is dependent 
upon a greater public awareness of what ment systems are, 
their perfection waits upon a demand that they be what they 
could be 

In Tfus Faith That Htalt, Sir William Osier, noblest physi- 
cian of his day, says, "Nothing in life is more wonderful than 
faith— the one great moving force which we can neither weigh 
in the balance nor test in the crucible " Through its faith in 
democracy this nation is philosophically committed to the 
merit principle Imperfections are transitory. The “sub- 
stance of things hoped for" w scientific selection of the beat 
qualified public servants 
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The chief function of a teacher is that of directing and moti- 
vating pupils toward the attainment of desirable educational 
objectives. In the performance of this function testing con 
play a vary important part When objectives are adequately 
defined and teBta devised which are valid with respect to them, 
the extent to which objectives are being attained can be mea- 
sured. Furthermore, such teats define the objectives for the 
pupils and motivate the pupils toward them When the results 
of such tests are adequately analyzed and interpreted, the 
teacher obtains a means of better orienting instruction and the 
pupils secure motivation through knowledge of progress 

Construction of exercises and analysis of the data resulting 
from their use ean make objectives more definite and more 
meaningful to the teacher foe creation of novel exercises 
and the analysis of data pertaining to them may widen the 
■cope of objectives recognized and ultimately realized in in- 
struction 

Instructional objectives are moat usefully defined in terms 
of observable behavior. Each specific objective should be an 
answer to the question "What should the pupils be able to do 
as a result of instruction?" Instruction which produces the 
abilities to do certain things, should concomitantly develop the 
attitudes, interests, and ideals which motivate their doing. 
Instead of the general and intangible objectives “good citizen- 
ship,” "appreciation of good literature,” and "scientific 
method,” specific objectives formulated In terms of observable 
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behavior may include, "Presenting arguments in support of the 
elimination of the general property tax based on factual evi- 
dence critically analyzed and evaluated," ''selecting a short 
story for leisure reading on the basis of the following cri- 
teria. . "rejecting a conclusion which goes beyond the data " 

Many instructional objectives must be concerned with fac- 
tual information, or knowledge In the selection of facta con- 
sideration should always be given to the contribution such 
knowledge cm make tp the types of behavior illustrated in the 
preceding paragraph The thinking necessary for adequate 
performance of activities recognized a a the really worth-while 
objectives of instruction must be based on knowledge. Infor- 
mation which makes no recognizable contribution to such 
thinking, or to the further learning which may contribute to 
such thinking, is not worth teaching. It is also not worth 
testing 

While teachers usually contend that their objectives are 
not reatneted to the memorization of miscellaneous, unrelated, 
and often trivial information, the tests used by most teachers 
are convincing evidence that their actual objectives are thus 
restricted When objective tests are made by teachere the 
exercises are most often of the true-false or multiple-answer 
types am) the content qf these exercises is wholly based upon 
information given in the text When essay tests are con- 
structed and used, the questions are factual in character and 
are scored only for the facts remembered. 

Tests should be designed which measure knowledge of facts 
True-false or multiple- answer exercises can be efficient means 
of measuring such knowledge. The possibility of represen- 
tative sampling of pupil knowledge is one of the advantages of 
objective testing In writing such exercises the teacher should 
be critical m the selection of the facts to be tested One useful 
criterion in the selection of facts is *he degree of relevance of the 
fact w some Important general concept nr principle which can 
be applied ip the solution of some problem involving reflective 
thought. Consideration should be given to whether or not 
klWwjedge of the specific fact contributes to further learning 
In many fields progress in leariung is contingent upon the syn- 
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thesis in the mind of the pupil of an ever growing body of 
factual knowledge In dunking about an important con- 
temporary social problem tho pupil may require a knowledge 
of numerous historical facts relevant to the trend which bu 
created the problem Senes of factual objective exercises may 
be useful in determining the extent to which such knowledge 
has been attained, 

In writing true-false exereises certain precautions should be 
observed Broad generalizations should usually be avoided, 
Such words at “always," “never,” “none,” “only," “all,” and 
“every" are obvious dues to the falsity of certain statements 
when the pupil con readily think of exceptions On the other 
hand, the statement "all echinoderzne live m exit water” rep- 
resents a difficult tiue-fslse item to one whose biological knowl- 
edge is not extensive. In multiple-answer exereises the incor- 
rect completions should not be too obviously incorrect. Each 
completion should be plausible It is frequently effective to 
wnte multiple-answer exorcises in which a "best" rather than 
e “correct” answer is called for. Such exercises tend to accom- 
plish more than the measurement of memorized information. 
All, or most, of the completions may be “correct"; the pupil 
must judge which completion is the "beat.” In a recent social 
science comprehensive examination the directions for one senes 
of multiple-answer exereises asked the student to identify the 
alternative suggesting the "most significant relationship” be- 
tween the two things mentioned. The first exercise in this 
senes is given below 

1, Minor party — social and economic reform' (A. Minor partus 
sre usually characterized by radical platforms; B minor parries sel- 
dom win an election, C reforms sgitatod by minor parties are oora*- 
tlmq adopted and enacted into law by the major partus, D untutv 
earful minor parties ultimately pass out of existence, E, the major 
artiea have the advantage in organization, funds, and prestige and, 
ante, ere more successful in promoting reforms ) 

While most of the answers are "correct," alternative ,f C 
represents the most significant alternative In u recent biolog- 
ical science comprehensive examination each exercise began 
with a statement to be explained. More than one of the alter- 
natives were “correct,” but only one was accepted ss the 
“explanation.” The following exercise is an examplei 
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Statrmant In embryologies! development certain itructuret de- 
velop id a manner analogous to the mnverewn of a passenger boat 
mto an aircraft earner (A The monila changea Into a Maatula 
which change* into a gastrula, D part of the placenta 11 formed from 
the wall of the uterui of the mother, C the mesoderm gives rue to 
the hearr, blood vweU, blood cells, lymphatics, kidneys, and certun 
other organs and structures, D in the higher vertebrate*, the middle 
ear cavity, the eustachian tube, the thymus gland, and the parathy- 
roid glands develop from gill aim or arches, E the ectoderm gtvei 
me to the epidermis and ro the fining of the mouth and anal aper- 
tures ) 

The answer "D’* is the alternative which best explains the 
ctatement. Thu example illustrates another very important 
matter If the teacher desires to measure how well pupils can 
handle questions involving thought rather than memory, an 
exercise must constitute a novel problem. It ii improbable 
that the students were taught the particular analogous relation, 
■hip implied in the above exercise. If they were taught this 
analogy, the crercue will measure only the extent to which the 
analogy is remembered If, however, they were merely taught 
the facts represented by alternative "D n then the relating of 
this alternative to the introductory statement becomes an act 
of thought transcending the utilization of memory. The ex- 
amples just given also illustrate the fact that exercises which 
are simple in form can be effective in the measurement of be- 
havior involving thinking. It is not necessary to construct 
exercises involving complex directions in order to obtain such 
measurement. "Hie important thing ib that the content of the 
exercise should be to some extent novel to the pupds 

Ducnmtnaiwe Thinking 

Oni; means of securing the novelty referred to in the pre- 
ceding paragraph is to bring together in a senes of objective 
items numerous facts whose classification in certain ways will 
involve the ability of the pupil to do discriminative thinking 
and to synthesize his knowledge. Let us suppose that pupih 
in physics have hcen taught certain facts pertaining to sound 
and, at a later date, certain facts pertaining to light Let us 
suppose further chat the teacher has not stressed the similarities 
and differences of sound and light phenomena The good 


teacher would probqbly stress these things, but For the sake 
of our illustration, let us suppose that this is to be postponed 
until the test has been given If this has been the case then 
die following senes of exercises should involve discriminative 
thinking and, as this thinking is taking place, require a synthe- 
sizing of knowledge by the pupil' 

On the line preceding each of tho following itanu, write the letter 
A if the item is true of sound 
7 d the Item is true of light 
C if the item Is true of both 

Its velocity in water is greater than m air. 

_ It can be reflected. 

It can travel through a vacuum. 

It can be refracted 

_ Etc. 

A number of examples of this very useful general type of 
exercise are given below from various fields In each case only 
a Tew of the items are listed 

In each fituackui below, an individual or a group of individuals is 
seeking protection or assistance On the line preceding each of the 
following items, write the latter 
A if the Federal government is responsible 
7 d the state government is responsible 
C if both governments are responsible 
B if neither government is responsible 

The New York Life Insurance Company wishes to open up 

agencies and tell insurance id Oregon, 

_ Mr. Jones receives in payment f 1,000 In bd(s which he presently 
learns are counterfeit 

— A Chicago visitor from Fort Wayne. Indians, stiffen severe in- 

juries when his car is wrecked because of defective pavement on 
76th Street , 

Convicts escaped from Joliet Penitentiary arrive m Des Moines, 

Iowa, bold up a bank, and ere seized and held by the local au- 
thorities. 

- Etc 

On the hne preceding each of the following items, write the letter 
A if the sentence is fragmentary 
7 if the sentence cun tarns a comma fault 
C if the sentence contains ■ dangling modifier 
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D if the sentence exemplifies lack or parallelism or faulty parallelism 
7 if the sentence is correct 

- One day I feel ae though I could lick the world, the next day I 
feel like a ewatted fly 

— - Upon returning from the store, my homework requires my 

attention. 

— As we walked along the hall, where a large photograph of Roose- 
velt hung. 

_ All of the boys being gone, moet of the manual labor waa done by 
the girls 

— Bemoaning her stupid lot, the army and navy claimed all of her 
fnends 

— A freshman learns to study with regularity, to play with enthusi- 
asm, and co-operauon 

— Etc 


On the line preceding each of the following Items, write the letter 
A if the statement is true of the Lyniai 
1 if the statement is true of the Adonau 
C if tire statement is true of both works 
D if the statement is true of neither work 

— The poem was wntren in memory of a dead fnend 

_ In form as well as content the poem displays ■ deep personal grief 

— The poet ornaments his verse with many classical allusions, 

— Etc, 

The following type of exercise can be used in a variety of 
school subjects and is effective in measuring how students can 
make comparisons,* 

On the hne preceding each of the fallowing paired items write the 
letter 


A if the item at the left of the page is of greater magnitude than the 
item at the right 

^ if the item at the right of the page is of greater magnitude than the 

— item at the left 

C If the two items arc of equal magnitude 

-- Amount of glucose in blood,. Amount oT glucose in the 

entering the liver 4 hours after blood leaving the liver 4 hours 

a meal is eaten after a mean* eaten. 


Amount of absorption of foods . 
by stomach. 

— ■ Percentage of urea in blood, 
entering the liver 


. Amount of absorption of foods 
by jntesnne. 

.Percentage or urea in blood 
leaving the liver, 
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— Amount of heat produced in. . Amount of heat retained In 

the body. the body ' 

— Etc. 

The same form needs only alight adaptation to be useful in 
writing chronology exercises in histoiy 

On the bne preceding each of the following paired items write the 
latter 

A if rho event in Column I occurred before the event rn Column II 
B d the event In Column II occurred before t^e event In Column I 
C if the events occurred at approximately the some time (within 
about a year of each other) 

Column I Column IT 

— Clayton Antitrust Act ....... . Sherman Autltniit Act 

— Alabama Claims Case .. .. Venezuelan Arbitration 

— Dred Bcott Decision ..... . Fugitive Slave Act 

— "Wigwam Convention" Fort Sumter fired upon 

— Etc. 

It should be mentioned that there should be some relation- 
ship between the paired events significant enough to warrant 
their being paired Care should be exercised when writing the 
"C” items to give events that oeeurred simultaneously, or very 
nearly simultaneously Note the qualifying remark with re- 
spect to category "C" in the directions stated above 

Exercises of similar form are useful in measuring the way in 
which pupils handle correlated or cause and effect relationship! 
In writing such Items where the relationship is definitely cause 
and effect, the cause should be given first 

On the bne preceding each of the folio -/in j paired items write 
the latter 

A if increase in one of the things referred to is usually accompanied ' 
by increase in the other 

B If increase in one of the things referred to is usually accompanied 
by decrease in the other 

C if one of the things referred to tends to remain the seme when the 
othet Increases or decreases 

— Amount of carbonates dissolved In the water of a river Number 
of dams in the nver. 

— Temperature of dip environment of a bird or mammaL Body 
temperature of the bird or mammaL 
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_ Amount of dissolved lift (sodium chloride) In * given body of 
water Nutnbcr ol amphibia m *h« water 

Extent to which men nuke changes m an area kite at which 

the ana tend* toward balanced equilibrium 

— Etc. 

On the line preceding each of the foDowmg paired item*, write the 
letter 

A if npiificsnt uicrerae In one of the thing* mentioned hai usually 
been accompanied by rngnifieunt inereue m the other 
B if ugmfirant increase m one of the thingr mentioned hai usually 
been accompanied by significant decrease In the other 
C if one of the thing* does not tend to change significantly when 
significant change take* place in the other 
Dollar diplomacy Confidence in the United States on the pair 
of the South American Nations, 

Effort* co eliminate the poll tax. Outcry an state right* by 

Southern senator* 

— Effort* co liberal ire the Supreme Court, Exprfcown* of oon- 
Hmtun with respect in the iuue on the part of the general 

public. 

Acceptance of the doctrine of check* and balance* Lobbying, 

_ Ett 

Inlslhgeni -Reading 

The ability to read intelligently in a field is an important 
general objective and more efforts should be made to measure 
how well pupils can perform this activity This ability is a 
very important factor in Anther learning Within a field and in 
desling with practical problems when formal school learning 
has terminated. The following exercise Illustrate* one means 
of measuring such ability. In this example, the material to be 
read is a graph. Such exercises need not be so restricted 
One may use paragraphs drawn from texts other than those 
studied by the pupil* and even from advanced texts if the 
teacher wishes to challenge the pupils. When paragraphs are 
selected they should be self-con earned in that the topic ts 
treated fairly completely. It I* frequently effective to present 
paragraphs Which give scientific data and some of the state-, 
menu luted may be inferences which may justifiably be derived 
from the data. Other* of the statements may be only partially 
justified or may be irrelevant. Some of the items of the faL 
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lowing exorcise represent predictions which may not be Jmd. 
fled* 



B. >f tiie Information given in the chart is suffiaont only to indicate 
that the statement Is probably true 
C if the information given in the chart is Sufficient Tor a judgment 
that the itatenunt n definitely folio 
D if tiie information given in the chart IS Sufficient for a judgment 
that the statement is probably falsa 
E if the information given id the chart is not sufficient to Indicate 
any degree of truth or falaiey in the sntomant 

_ Leu money was spent in 1930 than in 1929 for welfare and educ- 
tion. 


— In 1931 and 1932 the expenditure of money for highway purpose* 
was evidently conudered a means of combatting the Uoprwwon. 

- In 1940 a much greater proportion of the total expenditure* wu 
for welfare than in 1942 



— The increasing amount of money spent by the State for all pur- 
pwe* between 1929 and 1940 mu«t have come largely I nun 

or from Federal granu-m-aid rather than from borrowing. 

— Etc. 
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Such exercise* can be scored in the usual w ay to obtain a 
total score which show* how Well the student* agree with the 
key set by the teacher, or the "expert thinker" {n the field. 
Such a total score may answer the question* “To what extent 
doea the pupil read data accurately?” and “How well does the 
student recogoiee valid general (ration* drawn from data?” It 
is possible, however, to obtain other icon* which reveal certain 
characteristics of pupil tbmking. For example* a count of the 
number of M B” items merited "A*" and 4, D” items marked “C,” 
and M E" items marked with something other than “E" is in- 
dicative of the extent to which the pupil rends to go beyond the 
lata A relatively low score would be indicative of relatively 
{teater maturity in thinking with data. Such analysis is one 
af the characteristics of the handling of the test data in the 
Eight Year Study of thb Progressive Education Association 
and In the Cooperative Study in General Education of the 
American Council on Education, The following statement ap- 
peals in a recent report of the Board of Examinations of The 
University of Chicago. 


The modern tendency u to contruce the items so that the wrong 
retpoasra are wrong In a spsaficwajf— for eumpte, s definition when 
ts too breed, or e definition which u inadequate. When alternative 
■rawer* ore *o chosen, a regular pattern of incorrect res non w* i# 
established which upon analysis yield* much more information about 
the students’ mental haUu than did incorrect responses constructed 
by the older methods. By meant of this pattern type of analysis, It 
u possible to determine wnstbra ■roden t*' «rn» in reading md inter- 
preting date consist in saying that certain chugs are tn the dare 
which hi fact are sot, or in saying that certain thing* ora not in the 
data which in fact ere. It may he found that iruonta are able to 
»dett statement* which iame wub the dat* presented but have diffi- 
culty with statciBSnt* which disagree with the date presented- 

It I* difficult to wms exercise* 0 f the types described Such 
eumsei must be written and used, however, if pupils are -to 
seek worth-while objective* and If the degree of attainment of 
wprth-while objectives is to be minuted ^fhere measurement 
Is restricted, objective* are also restituted. 

'When a teacher has written such fctercjse* it U essential to 
recore careful ^valuation by other wachera who have the sun 
Itntttl objective*, The bwt evaluating u done when the 
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evaluating teacher doe* not have accesa to the key, but answer* 
tho items heroelf Comparison of several such evaluation* ii 
valuable in the rejection of bad items and in the revision of 
othen. For example, in a biological science examination the 
students were asked to mark certain items true or false on the 
basu of the "principles of inheritance M The following item 
wa* accepted as false by several biological science instructors; 
“Boy* tend to raaomble the father, while girls tend to resemble 
the mother" However, one instructor pointed out that in a 
certain important respect boys always resemble their fatbefs 
and girls always resemble their mothers In the direction* pro* 
ceding the items the following qualifying phrase waa added to 
take care of the situation! “excluding primary and secondary 
sex characteristics” 

The preceding paragraphs have dealt exclusively with ob- 
jective exercises In any balanced program of testing soma es- 
say exercise* should be included. In writing such exercise* 
“fact" questions should be avoided. Objective exercises can 
teat knowledge of feet* more efficiently and representatively 
than essay question*. Essay exercise* should represent novel 
problematic situations For example, the following essay ex- 
ercise appeared in a recent physical science comprehennve 
examination' 


in 14/2. wbn*topher Cohimbu* began me voyage of discovery by 
ssilini tourimost to the Canary Island* which arc near the coik of 
AFnca and in latitude 28* N. He then continued his voyage by nQ- 
JP rl“ p *** Atjan.de Oeean between the equator 
and 30* N On hi* return trip to Spain, early in 1493, he fintteiled 
northeast unnl he wa* somewhat more than 30“ N and than ailed 
cut to Spaut. On the basis of inform soon given In your phydtel 
science text, explain why Columbus aided as described above. 

Several blank lines followed this exercise In the test booklet 
Nothmg Is isld about Columbus /n the physical Science tot, 
but information on the belt* pf the Winds is given which could 
bo applied by the student in responding to the quettion Co- J 
lumbut took advantage or the northeast trades ui his voyage 
to the New World and of the prevailing westerlies on hu rctm* - 
to Spam. i| 

Eisays have been based on election* of quoted mateiul^ 
and on cartoons. In the field of English composition jt 
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effective to have mayi bwd on note* P«“ted tor reading 
at the time of the examination, or prior to the time of exanww- 

* ^Information is needed for the writing of a correct response 
to eway exercises which are thought questions or novel prob- 
lem*. but the quality of the reiponM will also depend upon the 
stent to which the student critically analyws the aquation 
and thoughtfully organic* ha information ui an effort to 
meet it. The scorers should be sensitive to more than thecor- 
nctneu of the information presented by the student. Their 
raU n g» should be based not only on the correctness of the facta, 
but also upon evidences of superior selection, evaluation, and 
organization of the facts presented. Comparison of student 
response* with a scale of response* to the same or to a annuar 
quMwm may be found to be an effective procedure. Another 
possibility is the use of directions for scoring m which such 
characteristics as organization and originality are defined and 
illustrated, and suggestions a remade with respect to the weights 
to he given to each soch characteristic 


Aiutyrii of Teil Scores 

After a tat has been given and scored the test data should 
be subjected to analysis Anslysis is essential if the teacher u 
to know the extent to which objectives are being attained 
One type of analysis has been referred to in the paragraph fol- 
lowing the exercise based on a graph It is also very desirable 
to determine the per cent of correct response to each exercise 
for the group taking the rest A low per cent or correct re- 
sponse may indicate the need of further instruction In some 
cases low per cents of correcc response may warrant the rejec- 
tion of such items lor use in testing subsequent classes, or the 
omission of *uch subject mutter as inherently too difficult. 
The analysis may be extended further than merely determining 
per cents of correct response. One can, with a little labor, de- 
termine how well each exercise correlates with whatever u 
measured by the teat as a whole. 

One way to do this type of analysis is to separate the papen 
into two groups The “upper group” contains all papers above 


TEACBba TESTS 

the median .core el the test .« a whole wMe the "lower group" 
contains sll pepeiewith totsl stores below the median rntal feat 
more Tailing one Celt paper at a time and oppoaite the num- 
ben oi the exercises on a tally sheet, the tescher tallies lor each 
exenhie correctly answered For example, if Ihe fine paper 
has correct answers for exercises 1, 2, S, 7, and so on, the teacher 
makes e tally mack after these numbers on the tally fom 
Another foim h similarly prepared for the “lower group Ihe 
per centl ef correct response for each of the groups are then 
computed, (Semplea of 100 paper, m each group svoid the 
necessity for tuch a conversion,) Since there are equel num- 
' here of papsre in the upper and lower groups correspon ding per 
cents may be averaged to obtain the par centl of correct re- 
iponse for the entire group taking the teat The pet cuts i lor 
the upper and lower groupi are used in reading the correlation 
coefficient from the abac shown on page 122 

For example, suppose that 65 per cent of the upper group 
answer exercise 17 correctly while only 25 per cent of the lower 
group do K The correlation between suceese or failure on the 
item and the total score on the rest is + .60 Such «n item 
makes s significant contribution to whatever ia measured by 
the ten ea e whole. Thla correlation can be seen in the follow' 
mg table which need not be conetnicted for each item, but 
which u useful in explatmng the above 

FAHUWe ON in* SUCCESS On TTM 


35 

6S 

75 

25 


Taka the per cents 80 and 55 aa another example. Here 
the intellect does not foil on one of the lures in the chart. One 
mint interpolate between the lines labeled r«+ .40 and r*= 
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+ ,50. The r m ihi* caae u approximately + 43 (One inter- 
polates along the imaginary line perpendicular to these curve*,) 
When an exercise drops below + 20 in correlating with the 
total teat score the inference is that the exercise is nor making 
a significant contribution to the test The first time n care- 
ABAC FOR ITEM-TEST CORRELATION 


mCSNTABS OS LOVU MOOT r AMINO TKL I rod 



fully con*nictcd ie*t is given it is noc unusual to find that one- 
fourth or one-thud of the exercises drop below thu value. 
Certain exercises may yield negative correlation*. For ex- 
ample, suppose chat only 25 per cent of the upper group an- 
swered ekercue 4S correctly while 65 per cent of the lower group 
responded correctly. The correlation is — ,60 and this nega- 
tive relationship is illustrated by the following table* 


Main ON mil SUCCESS ON ITEM 


ABOVE MEDIAN 


BELOW MEDIAN 


Low or negative correlations most often indicate that an 
exercise is bad The key may be wrong, the exercise may be 
ambiguously stated, or the exercise may bo much too easy or 
much too difficult: Analysis of responses to other sfrematiyei 
can determine a better key when the exercises are of the type 
requiring judgments; for example, the type of exercise illus- 
trated with reference to the reading of the graph of state expen- 
ditures In some cases a low or negative correlation doe* not 
necessarily mean that the exercise Is a poor one. It is possible 
that the exercise is measuring some trait which is worth while m 
itself, but which is not related to whatever is measured by the 
test as a whole In every case the teacher should study the 
exercise in relation to the total per cent of correct response and 
the correlation coefficient, and should formulate judgments with 
respect to the merit of the exercise and the degree of attainment 
revealed by the data. 

The type of analysis just described applies to objective ex- 
ereues No definite procedure can be suggested far essay ex- 
ercises When the scores on a given exercise range over several 
points, ordinary Pea non product moment coefficients can be 
calculated between the score on the exercise and the score on the 
teat. It would seem much more useful, however, to analyze 
essay lesponsa in terms of some classifies non with respect to 
various types of merit or of limitation*. 

The discussion in the preceding paragraphs may seem to be 
carrying the ‘'improvement of testa 1 * a hit too far for the dais- 
room teacher. It may seem that an Inordinate amount of work 


IS 

25 

35 

65 1 
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and cerebration is invoiced If, however, teaching ip as much 
worth doing aj the producing of manufactured articles or mate- 
nals where the processes are controlled from stage to stage, 
then the lahor involved js more than justified The teacher 
who is willing to do these things will find that the “improve- 
ment of tests" is not merely that. It is also the improvement 
of teaching. 


PREDICTION OF COLLEGE SUCCESS BY MEANS OF 
THURSTONE'S PRIMARY ABILITIES TESTS* 


CHARLES a GOODMAN 
The Pmniyivula Sqm CoUrga 


In his 1940 monograph on factor analysis, Wolflo (18) 
states: "Few attempts have been made to put the result* of 
factor analysis to practical use Most of these few have dealt 
with intelligence testing . Some studies, Chern (5) and 
Schneck (9), have indicated that achievement in individual 
courses can sometimes be predicted as well [as], or better, by 
special tests or by specialized subtests than by the total score 
of a general Intelligence test. Which tests will beat predict 
grades in individual courses is an empirical problem As a 
start toward answering that problem, Thu rs tone hu prepared 
a battery of sixteen tests, giving perception, number, verbal, 
space, memory, induction, and deduction scores (15). He 
warns (16) that this battery n still in the research stage and » 
not ready for routine use. It has been used by Bemreuter (3) 
and Stalnaker (11), but no data have been published to indi- 
cate how well It answers the administrator’s wish for hotter 
methods of predicting scholastic achievement.” 

This paper is a report upon the studies conducted at The 
Pennsylvania State College specifically related to the possi- 
bilities of unrig the Thuretono Primary Abilities Texts for the 
purpose of predicting scholastic achievement, it provides, in 
some measure, a partial answer to the statement made by 
Wolfle (IS) that there is a need for data to indicate how wd] 
these tests answer the administrator's wish for better methods 
of predicting scholastic achievement 

'Tbc writer wishes a scknmtcdsc ha flunks and tpprxcatfcn tn Minus* 


report lbs Endup rf (liar atiniw related tn tn paper 
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Experiments end Findings 

Bemreuter and Goodman (4) conducted a study in 1939 
to determine how well they could predict the success of fresh- 
men engineers at The Pennsylvania State College by means of 
the Thun tone Primary Mental Abilities Tests. One hundred 
seventy freshmen engineer! served as their subjects. Bem- 
reuter end Goodman then obtained correlation* for the (our 
college courea of Oiemistry, Drawing^ English Composition, 
Mathematics, and semester point average* with the Primary 
Abilities Tests. They then calculated multiple correlations, 
using various combinations of the abilities, with semester point 
average, Chemistry, English Composition, and Mathematics. 
Ths results are shown m Table 1. 


+ .49. In both cases the increase in the size of the correlation 
is slightly more than that obtained uarng the single abilities of 
reasoning with Mathematics, + 44, and verbal ability with 
English Composition, + .44 The best multiple correlation ob- 
tained, + .51, was with fint-semeater point average, using the 
abilities of number, verbal, space, induction, and reasoning. A 
more detailed report of this work has been published in ui 
earlier paper by Berarcutor and Goodman (4) 

During 1940 Ball (l) administered tho Thunrtone Primary 
Mental Abilities Tests to a group of 147 female freshmen and 
159 male freshmen attending the Liberal Arts School of 1)10 
Pennsylvania State College He then correlated their ten 
■cores for each of the seven primary abilities with the college 


TABlfcl 
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The highest single correlation obtained was + ,44 for reason- 
ing with Mathematics and + 44 for verbal ability with English 
Composition By using a combination of the number, space, 
induction, aod reasoning abilities, and calculating a multiple 
correlation with Mathematics, they were able to obtain a mul- 
tiple correlation of +.49. In tbe case of English Composition, 
by combining the abilities of number, verbal, memory, Induc- 
tion, an d reasoning, they obtained a multiple correlation of 

isrw* csTsw^ncs stesswsi'&raS' • 


Senate post lysate,, 

Ait 





coursea of Art, Botany, English Composition, French, History, 
Mathematics, Physical Science, Political Science, Zoology, aid 
with their firat-feruester point average. Ball's finding! ire 
shown in Table 2 

It will be seen from Table 2 that the range of correlations 
found by Ball is from -.12 to + .41. The highest correlation 
obtained was + 41 for number ability with Mathematics The 
next highest correlation, + .40, was that of verb*! ability with ' 
English Composition. Verbal ability correlates more highly % 
and positively with each of the courses than does any of tb* Jt 
other abilities. The highest correlation Ball obtained for pjjy 
of the abilities with Semester Point Average was + .35 with 
bal ability In u further effort to determine whether he cOuldii 




128 EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 


PRIMARY ABILITIES TESTS 


129 


increase his predictive possibilities with the Thuratone teste, 
Ball computed a multiple correlation of the memory, number, 
verbal, induction, and reasoning abilities, using as his criterion 
semester point average The multiple correlation was + 46. 
Dali's results show that optimum weights yielded only a slight 
increase in correlation over that obtained for cha Bingle factor 
of verbal ability, which correlated + 35 with semester point 
average 

Efeuemer (8), in a study similar to Ball's, attempted to 
determine the predictive possibilities of the Thuratone tests in 
the School of Chemistiy and Physics at The Pennsylvania 
State College. Upon administering the AbdttUs Tests in 1942 
to 147 freshmen students, she correlated their test scores on the 
pnmaiy abilities with their first-semester point average and 
with the course of Inorganic Chemistry which each of her sub- 
jects had taken Inorganic Chemistry at The Pennsylvania 
Stare College is the first course in chemistry taken by freshmen 
students It is both intensive and extensive in scope, as can 
be seen from the following description in the College catalogue* 

Inorganic Chemistry (5).— The non metallic dements, fundamen- 
tal pnnO[>l«* of the science arc studied In connection with the deicnp- 
tree Chemistry of nonroetalllc dements and their compounds, prepares 
for future study of the science. Lecture 2 hours, recitation 2 hours, 
pncticum 3 hours 

The results obtained by Hessemer are presented m Table 3 
and show, as did Ball's results, verbal ahility to be the best 
single predictor of semester point average. The best positive 
correlation, + .18, was that of reasoning with Inorganic Chem- 
istry This correlation is slightly larger than four times the 
size of its probable error. Interestingly enough, the largest 
correlation found was negative, - 2S, for space with Inorganic 
Chemistry. When each of the primary abilities wm correlated 
with semester point average, the highest correlation obtained 
was that of + 44 with verbal ability, while reasoning correlated 
neat highest, + 40 

In 1941 Trediclc (13) conducted a rather thorough study 
of the predictive poaabilities of the Thuratone Primary Mental 
Abilities Tests and a battery of vocational guidance tests. Her 
guidance test battery consisted of the Otu, the Pressey, the 
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Minnesota Paper Pom Board, the Minnesota Clerical Test, 
the Meter-Seashore Art Judgment Test, the Minnesota Assem- 
bly, and the Minnesota Spatial Relations Test , Her subjects 
consisted of 113 freshmen women in the Department of Home 
Economics at The Pennsylvania State College. After adminis- 
tering these teats she correlated the 1 seven abilities with the 
grades made by her subjects on the five college courses of Art, 
Chemistry, English Composition, Home Economics 101, Home 
Economics 109, and fi rat-semester Point Average. Her find- 
ings are shown in Table 4. 

It will be observed from Table 4 that the range of correla- 
tions is from - 02 for memory correlated with Art, to +.55 for 
verbal ability correlated with English Composition. It will 
be noted that Tredick's results, like Ball’s and Hcssemer's, 
showed verbal ability correlated more highly with each of the 
courses than did any of the other abilities The highest any of 


TABLE 4 
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these abilities correlated with Semester Point Average was + .51 
and again it was verbal ability. 

Tredtck then calculated the correlation# of the battery of 
vocational guidance tests taken by her subjects with the same 
five college courses which were used to obtain the correlations 
with the Thuratone tests, and lemesrer point average. The 
findings are shown in Table 5 and afford some interesting com- 
panions with those obtained with the Thuratone A biltties Tests 
TABLES 


ftmUNw rf d* ftuwil ChUmm BtUtry CttU v Oredu mJ Simitur 

PoM Av*rwga m ffm Aaumla 
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The highest correlation obtained for a single test of the 
vocational guidance battery with college grades was that of the 
Otis (Higher Examination, Pom A), which correlated +.54 
with En glis h Composition, The next highest correlations ob- 
tained were for the Otxs with Home Economics 101, + 4B, and 
the Pressey with English Composition, +.48. It will be noted 
that both of the group tests for mental ability, the Otu and 
Pressey, correlate higher with these college courses than do any 
of the other tests of the vocational guidance battery. Further- 
more, the Otis correlates more highly with Semester Average, 
+ *53, than do any of the other rests of the vocational guidance 
battery. The Pressey correlates only slightly lower than the 
Otis with Semester Average, +.51. Comparing Table 4 with 
Table 5, it will be seen that the verbal factor la the only ability 
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that appeara to correlate as well as the Otis or the Pressey Tests 
with the various courses The Otu correlates slightly higher, 
+ .53, than does verbal ability, +31, with semester point aver- 
age. However, the correlation with semester point average of 
the Pressey is the same, + Jl, as that for verbal ability 

In an effort to determine the relationship between her sub- 
jets’ scores on the Primary Abilities Tests and their scores on 
the vocational guidance battery, Tredick calculated the corre- 
lations for each of the tests Her results are shown m Table 6, 
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The data of Table 6 offer some indications why the correla- 
tions of Thuratone’i verbal ability With the five college courses 
are so similar in size to those correlations obtained for the 
Pressey and Otis with the same college counts. Of the seven 
abilities, verbal ability has the highest correlation of the entire 
table, + .76, with the Otis Test The next highest correlation, 
+ .68, also involves verbal ability with the Pressey It would 
appear from the correlations that both the Pressey and the Otis 
contain materials similar to those found in Tbumono’s verbal 
ability It also seems that the materials of the reasoning, in- 
duction, and the perceptual factors overlap the materials of 
the Otis and Pressey Tests, w can be seen from the correlations. 
It may be that the perception ability {a related to the "speed 
factor found to operate in tuned tests such as the Otis and 
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Preise y Memory ability correlates only slightly with the Otis 
and the Pressey, Induction and reasoning abilities correlate 
highest with the Revised Minnesota Paper F orm Board, The 
primary abilities appear to correlate only slightly with the 
Minnesota Assembly Test, The Minnesota Spatial Relations 
Test correlates best of all with perception ability, but space and 
induction also appear to overlap to some degree with this test. 
On the Minnesota Clerical Test, Name Checking correlates 
highest with the perception and number abilities. On Number 
Check tug, the five abilities of verba], space, memory, induction, 
and deduction do not appear to be related On the other hand, 
fourof these five abilities, with the exception of momory, appear 
to be related. 

Finally, Tredick combined the abilities of number, verbal, 
induction, and deduction, using as her criterion semester point 
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average, and obtained an Jl of * .61. Similarly, she combined 
tho Otis, Prtssey, Minnesota Paper Farm Board, and Number 
Checking of the vocational guidance battery, using as her cri- 
terion semester point average, and obtained an R of + 57. 

Goodman (7) made a second attempt in 1940 to determine 
the predictive value of the Abilities Test in determining college 
success for his engineer subjects at the completion of their first 
year in college Each of the factors was correlated with the 
first yea rig gomes tor point average. The result! are shown in 
Table 7 

The abilities of reasoning, induction, verbal, apace, and 
number were then used in a multiple correlation with first 
year's semester point average and the combined variables 
yielded a multiple correlation of + 49, which is slightly lower 
than that of + 51 obtained for the first semester 

Goodman (7) then sought to determine whether be could 
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obtain better predictive values for his engineer subjects by 
using the sixteen Thurstons teats individually. These teiti, 
when combined, yield the score that is the measure Tor the abil- 
ity. According to Thuratone (14), each of the testa designed 
to measure the particular ability is highly saturated with that 
ability. The correlations between the testa that measure three 
abilities are shown m Table 8, The teats of the three abilities 
of number, verbal, and space appear to correlate highly with 
each other, while the correlations of the teats of the other abili- 
ties descend in size. 
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The abilities tests were then correlated with the first year's 
semester point average and the zero-order coefficients were ob- 
tained. For the purpose of comparison, the zero-order coeffi- 
cients for the teats and the ability zero-order coefficients t re 
given in Table 9 The following facts are to be notedi both 
testa or the perceptual factor correlate only slightly with the 
criterion as does the factor itself, but it can be seen that the 
Identical Forms Test alone correlates higher than the composite 
of the ability tests, while the correlation for the Verbal £niim/r- 
ation Test is low in value. In number ability the r for the 


134 EDUCATIONAL mil) PSYCHOLOGICAL MEASUREMENT 


PRIMARY ABILITIES TEATS ' 135 


single teat of Addition is as high as the r for the factor itself and 
the Multiplication Test r is not much smaller In the verbal 
ability, the Completion Test ru again almost as high aa the abil- 
ity r. In apace, the correlation for the tat of Cards alone ia 
greater than that for the ability, while the r for the second 
teat of Figures is of low value. Both tests of memory cor- 
relate very slightly with the criterion while the r for the ability 
itrelf Js slightly greater than the r for the Word Number Test, 
The three tests of induction have ria of approximately the same 


TABLE 9 

ZnuOrim * el lb Sum TkunUnu Tuti wUi tie Crtionm 


Fiew 

Tat 

Fictar r 
*bh aitenun 

Tac r wnh 

P 

I denial Farm 

YaW Esnmnnon 

+JB 

+J» 

+ J0 

N 

V 

S 

Addition 

Mnicpfatwa 

CampleUJ* 

SwveOppw** 

Oudi 

Firm* 

+M 

+44 

+ II 

+4« 

+44 
+ JI 
+ 40 
+43 
t 10 

M 

Word Nwnbcr 
lunik 

+41 

+ 10 
+ B8 

I 

1jO« Groapinf 

Number Fillers 

+41 

+45 

+ 40 
+44 

D 

Anduuik 

Number Sena 

Mnfawcal blori^m* 

+4fi 

+J2 
+42 
+ 10 


mat, while tba r for the ability ik somewhat larger than that for 
any of the tests In reasoning, tho r for the ten of Arithmetic 
is almost as large as the t for the ability, with the Number Senes 
Test r slightly smaller. The tart of Mechanical Movement/ 
correlates slightly with the criterion and with the other two 
tats of the ability, Upon obtaining the zero-order coefficient! 
between the teats and the criterion, u rnjdtiple correlation of 
eleven of the tests was computed The cloven tern were 
rejected which correlated highest with the criterion and lowest 
with, the other rests. The tests selected were Arithmetic, 
Number SitUr, Some-Opfasiies, Completion, Leltrr Groups 


mg, Addition, Marks, Number Patterns, Cards , Multiplication, 
and Word Number The multiple correlation obtained, using 
these eleven tests, was + 48 

It was apparent that there whs little hope of raising the 
multiple coefficient by adding the remaining five tests, since 
they correlated very little with the entenon It will he noted 
that with the eleven tests the correlation was no greater than 
that obtained by using only five of the abilities 

White (17) in 1942 made a study of the Thurstone tests 
and college prediction in Home Economics, using 94 1 of die 
same 113 subjects used by Tredick. At the time Tredick con- 
ducted her study these subjects were freshmen and had com- 
pleted their first semester's work. When White conducted her 
study, the 94 subjects she used of Tredick’s group were then 
sophomores. White combined the grades for all of the Indi- 
vidual courses taken by her subjects and took as the repreten- 
tative scare the students 1 avenge for each of the particular 
types of courses. Semester point average in White's study 
represent* tho average of two yean of college work by her sub- 
jects Correlations were then obtained between each of tbe 
primary abilities and tho avenge scon for work in Art, Science, 
Home Economica, and semester pomt average. The results 
are shown in Table 10 

It is worth comparing the results obtained by White Si 
shown in Table 10 with those qf Tredick in Table 4. White's 
data showed that each of the abilitiee correlated lower in sD 
cates but one with semester pomt avenge than those obtained 
by Tredick. The exception is reasoning ability, which corre- 
late! 4- .45 with semester pomt avenge and indicates an increase 
over the correlation of + 42 obtained by Tredick, 

The correlations in White's study for the abilities with Art 
average am generally higher than those found by Tredick* 
For Science avenge the ris of the abilities calculated by White 
are mixed, some being higher and others lower than those 
Tredick obtained for the abilities with Chemistry White's 
correlations are all higher with English avenge than those cil- 
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ciliated by Tredick, with the exception of induction, which is 
(lightly higher for Tredick All of White’s f's with Home Eco- 
nomics average are lower, with the one exception of induction, 
when compared with Tredick's correlationa for Home Economics 
101 and 109 Another fact worth noting is that in White’s 
itudy all of the correlations of reasoning ability aro higher than 
thou obtained by Tredick. It may be that reasoning ability 
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becomes more important when, presumably, the degree of diffi- 
culty of college work increases One mure fact worth noting 
u that White's correlation of verbal ability with English aver- 
age was + .65, while Trediclc obtained an r of + J55 for verbal 
ability with English Composition 

White then correlated each of the sixteen Thurstone tests 
with the average Tor each of the four courses and semester pome 
average. These results are also shown in Table 10. Twenty- 


two per cent of the correlations for the single teats in Table 10 
are higher than, or equal to, the correlations for the ability 
In many cases the correlations of the tests of the ability are 
only slightly smaller than correlation for the ability Finally, 
White computed multiple correlations, using first, combina- 
tions of the abilities, and secondly, the individual teats When 
she combined number, verbal, memory, induction, and reason- 
ing abilities with semester point average, she obtained a mul- 
tiple correlation of + 59 Combining number, verbal, and 
reasoning abilities and correlating them with semester point 
average, she also obtained a multiple correlation of + .59) 
Lastly, she combined the five individual teats of Completion, 
Same-Opposites, Arithmetic, Number Senes, and Addition, and 
obtained a multiple correlation of -i- .62 

Other Thurston a Studies 

A number of other studies have been reported on the pre- 
dictive possibilities of the Thurstone Primary Mental Abilities 
Tests. Yum (19), in istudy at the University of Chicago, cal- 
culated multiple correlations using various combinations of the 
abilities with semester average The best multiple correlation 
he obtained was + ,422 using all of the abilities, Shanner and 
Kuder (10) have reported correlations of the abilities with 
average grades for tho 1938 freshman class of the University 
of Chicago Tlie highest correlation reported for average 
grades was with verbal ability, + .415. Correlations are also 
reported by these writers for the abilities with four introduc- 
tory couracs at the University of Chicago Deduction corre- 
lated with Biological Science, + ,418; verbal ability, + 472 with 
Humanities, deduction, + 485 with Physical Sciences, deduc- 
tion, + 427 with Social Sciences Multiple correlations using 
all seven of the abilities with the four introductory courses 
yielded the following H'a, + .500 with Biological Sciences, + 541 
with Humanities, + J61 with Physical Science*, + 566 with 
Soda] Sciences, ( 

In 1941 Ellison and Edgerton (6) tested a group of 49 stu- 
dents at Ohio State University with the Thurstone Primary 
Mental Abilities Tests, The highest correlation obtained, 
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+ 44, was for verbal ability with point hour ratio The remain- 
lug ability correlations with point hour ratio range from - .24 
to + 51, Combining the seven abilities, they obtained an R 
of + ,640 with point hour Tatio Ellison and Edgerton also 
report correlations for the abilities with grades in college sub- 
jects The highest correlations reported were, verbs! ability 
with English, + 75, verbal ability with Science, + 68; induction 
with Foreign Language, + 78, reasoning with Psychology, + .63. 
They state "that the results can only be taken as suggestive 
and not as facts from which broad generalizations may be 
drawn” 

On the basis of the studies reported m this paper, the fol- 
lowing conclusions appear to be justified. 

I The Thurstone Primary Abilities Tests correlate, on the 
whole, hh well u most standardized intelligence tests with cri- 
teria of college success. 

2. The Thurstone Primary Abilities correlate with indi- 
vidual college courses to same degree and can be used for pre- 
diction of success in these courses 

3. Verbal ability correlates higher than any other of the 
abilities with semester point average and individual college 
course* 

4 Multiple correlations obtained by using various combina- 
tions of the primary abilities yield some increase m correlation 
over those obtained by using single abilities, when correlated 
with semester point average. 

5. Multiple correlations using various combinations of the 
■ingle tats that measure the primary abilities yield correlations 
wnh semester paint average that were in some cues higher than 
or equal to those obtained using the abilities 

6. Verbal ability correlates highly with tho Otis and Press*, y 

tests, and appeara to be overlapping some of the functions in 
these general intelligence tests. * 

7 The Otis and Presjay tests appear to contain somo of the 
same (unctions as those measured by the seven Thuratone pri- 
mary abilities, 

B. The single tests measuring an ability in some instances 
correlated highly with each other. 


9 A single test of an ability will in some instances correlate 
higher with the criterion, than does the composite of several 
tests of the ability itself. 
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TEST CONSTRUCTION IN PUBLIC PERSONNEL 
ADMINISTRATION 

DOROTHY C. ADKINS 


Introduction 

When the public is persuaded chat positions in the public 
service should be filled by the beat qualified persons and a- 
presses its conviction through a civil service law, a tremendoui 
responsibility, that of predicting which persons actually are the 
best qualified, devolves upon the agency charged with adminis- 
tering the law Increased emphasis on the impartiality of the 
selection of public officials hu been accompanied by growing 
reliance on examining processes that are objective. Hence, 
the major attention of this article is devoted to topics relating 
to the construction of the written examination in civil service 
In this interpretation, probloms common to the academic set- 
ting have been largely excluded. Although the article is further 
restricted to problems arising in state civil service or merit sys- 
tem jurisdictions, certain of the comments may apply equally 
to civil semes at the federal level 

However obscured it may be in practice, the essential of any 
merit examination is that it predict efficiency on the job Those 
who are not likely to perform satisfactorily on the job should 
be excluded from the final list of eligibles, and those who achieve 
places on the register should be ranked in the order of predicted 
job performance. To these ends each part of the total examina- 
tion process should contribute 

State civil service examinations often include, in addition to 
a written teat, a rating of training and experience and an orrf 
interview If proficiency in the operation of machines or equip, 
ment is essential, a performance test may be one of the com- 
Hi 
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ponents, Far some positions phases of the examination may 
well be modified or omitted, but usually not the written 
examination 

/fating of Education and Experience 

Most stste jurisdictions determine initially who shall be 
admitted to examinations by prescribing minimum qualifica- 
tions of education and experience. Within limits, additional 
education may be substituted for a part of the experience, and 
vice vena, The process thus screens fint those candidates 
who do not meet the minimum qualifications. Those barely 
meeting the education and experience requirements may be aa- 
slgned the lowest passing score and those surpassing the mini- 
mum requirement! higher ones, the scores depending upon such 
factors as amount, pertinency, and recency. Hie inclusion o{ 
ratings of training and experience as one pare or the examina- 
tion process is based on the assumption that differences among 
the candidates on these factors wQl be reflected in job per- 
form ence. 

Hie argument that the entire burden of ths examination 
process should be placed on such ratings is untenable, however, 
since the rating procedure is relatively subjective and unreliable 
and fails to tike cognizance of variations in the degree and ex- 
tent of knowledges and abilities that const even among those oT 
similar training end experience. Such variations often an 
of grta ter significance in the prediction of job performance than 
are differences among candidates in education and experience 
beyond the entrance requirements Further, if these require- 
ments are relatively high, differentiation among candidates 
meeting them may, by means of disproportionate weighting of 
mere survival, give undue advantage to those who have faded 
to progress. 

Thus If, in relation to the salary level and labor market, 
minimum requirements for a professional class ore low, say col- 
lege graduation, then assignment of a high educatioe-expen- 
PWrt to a candidate with two year? of pertinent graduate 
*bd three yean of closely relared experience probably 
W<wld Contribute to the validity of the total examination But 
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if for a similar class entrance requirements are higher, say two 
yeari of pertinent graduate education and three years of opul- 
ence, then giving ■ higher score to a candidate with five yean of 
experience than to a candidate barely meeting the minimum 
qualifications may be of questionable value, The principal 
effect of such practice in this instance might be to give unwar-, 
ranted higher scone to candidates who, although older, have 
not advanced beyond younger candidates also meeting the high 
requirements for the class Moreover, the rating of training 
and experience does not provide an evaluation of personality 
differences ' 

Tk Oral Intervltte ( 

The oral interview, despite its recognized limitations, seems 
to be the brat available instrument for appraising personality 
characteristics For civil service examinations little or no re- 
liance can be placed on papor-ind-penefl teats of personality, 
which would fad to elicit frank answers in a competitive situa- 
tion with jobs at stake Behavior in an oral interview can alio 
be faked. For this reason, se well as unreliable rating, the oral 
interview included In the final score is usually weighted consid- 
erably less than are the other two parti, even though it con- 
tributes positively to validity, and the number of candidate* 
failed on the oral p very small, 

Warranting at least passing mention are certain miscon- 
ceptions of what constitute desirable purposes of an oral inter- 
view, such u the idea that its aim u to reappraise the appli- 
cants education and experience or that it should he designed to 
ascertain the scope of the candidate*! knowledge or the degree 
of his general intellectual abilities, Since snch factor* can be 
measured more adequately by the other parts of the examina- 
tion process, the inclusion of a relatively unreliable rating of , 
them in the total score probably serves to lower not only thq 
reliability of the total composite hut, more critically, its 
validity. 

The Performance Tut - 

Where differences in skill in operating machines is a perjl*, ' 
ndit factor, the total examination usually Includes a perf^nD^J 
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uice tell For positions requiring such skill, the oral Inter- 

* is of negligible importance if the duties typically entail 
irttle contact with the public or with fellow employees Be- 
cause of this and not because they serve similar purposes, per- 
formance testa and oral interviews are rarely used for the same 
class Differential ratings of training and experience arc alto 
commonly omitted for machine-operator classes Performance 
tesu are costly, difficult to administer, time-consuming, and 
frequently n* very reliable. But their use may add appre- 
ciably to the predictive efficiency of the composite. In view 
of their limited reliability and their failure to take adequately 
mlo account the ability to profit from further teaming and 
apenencc, they are more appropriately regarded as a supple- 
ment to rather than a aubstituta for a written test. 

Tka Written Test 

Tart Content. Even though the other parts of the total 
xaminsuon process may be BQ important for certain classes 
that they should not ba dispensed with, no other is so significant 
or should bear bo much weight as a well constructed written 
elimination, The purpose of the written teat is to determine 
nlubly the extent of individual differences m pertinent areas 
of knowledge and abilities Defining areas of knowledge and 
the abilities to be, sampled in an examination is achieved by 
means of job analysis, the results of which are commonly sum- 
marized in a cl os 9 specification Such a compendium, depend- 
ing Upon its thoroughness and clarity, gives a picture of the 
duties of the class of positions, qualifications in terms of 
education and experience, the supervision exercised and re- 
ceived, ind the knowledges and abilities bearing on the duties 
of the class, This information should be supplemented by 
knowledge of the relationship of the class to the total organiza- 
tion and of the applicable salary range First-hand acquain- 
tance with the job, although unfortunately not always avail- 
able, is of inestimable value. Since the written examination 
must contribute to the prediction of efficiency on a particular 
type of job, its most effective construction it contingent upon a 
clear idea of what the job is. Its subject matter should first of 
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interpret the term practical to mean what they think the public 
meant by practical, A “practical** examination, then, becomes 
one that looks practical to the lay person, and, particularly, to 
the candidate An examination may of course be satisfactory 
valid from the point of view of its correlation with job perform- 
ance and yet fail to meet the criteria for “face validity” of 
thia type, In the interest of fostering public support of the 
civil service principle, construction of examinations efficient in 
predicting job performance and also acceptable tn the public 
as practical is the desideratum. An examining body that limits 
itself to the kind of face validity under discussion is not in any 
seme adequately fulfilling its funccion; nevertheless, its general 
aim will in ihc bng run be appreciably advanced if it achieves 
discriminating examinations that, at the some ume, have this 
type of validity 

Rigorous application of this criterion of practicality requires 
nDt only careful scrutiny of the broad areas of knowledge and 
abilities sampled in the test as a whole, and of the general area 
covered m each mm, but also attention to the several in- 
dividual concepts contained in each item Sometimes attenuon 
mint he focused on the individual word In a teat composed 
of multiple-choice items, for example, the candidate may not 
recall merely the question or just the question and the best 
anawer-H>bviously,he may not know which answer is intended 
to be beat But he may remember and criticize tome of the 
answen Intended to he ‘'distrSctera,” “decoys," or “confusions/ 1 
ds they ire variously termed. 1 IF an item constructor inno- 
cently includes Socrates in a diBtrscter, he should not be sur- 
prised if the examination is later publicized as absurdly 
unpractical because it inquires into the candidates' Greek 
philosophy Particularly ip (ho cose of civil service agencies 
not far removed from the publics they serve, every examination 
Rem should be reviewed from the standpoint of how it might 
look m the public prints 

Speed vtrfw Power Public opinion also has important 
bearing on other aspects of the examining process. For ex- 
ample, a ]■ more favorable to a power test than to a test tha 
places i premium On speed,' People say job duties are not per 


all be related to the prediction of job performance; and the 
weights to be assigned different areas of subject matter should 
depend upon theur contribution to the prediction of efficiency on 
the job 

The types of subject matter included in a teat have all too 
often been limited by lack of facilities essential (or item con- 
struction Administrators and test technicians place too little 
reliance on persons thoroughly familiar with the subject matter. 
Test technicians sometimes delude even themselves into behov- 
ing that they can construct or at least assemble an adequate 
examination in a specialized field without assistance. Much of 
the criticism directed at written tests is attributable to action 
based upon just such misguided self-confidence. On the other 
band, examining agencies should not vest sole responnbOity 
for either item construction or assembling examinations m 
subject-matter consultants who are not skilled in examination 
techniques The sounder and more successful approach as- 
sumes collaboration between those schooled in content and 
those vened in technique, although an occasional agency may 
be fortunate enough to have on Its staff a penon thoroughly 
competent m both a subject-matter field and examining pro- 
cedure* 

The content of the written examination should be related to 
realistic class specifications It is by necesnty limited by the 
facilities for constructing or securing items A third factor 
that ennnot be ignored in its bearing on examination content 
is public opinion. Almost universally, civil service examina- 
tions are required to be “practical and related to the job,” al- 
though the exact statement of the criterion vanes It would 
■cem at first thought that establishment of a "satisfactory” 
relationship between test scores and performance on tho job 
would guarantee the meeting of dm criterion— and the menu 
of such an argument will not be denied. Pertinent to this con- 
tention, however, is the difficulty of establishing that the corre- 
lation of an examination with job performance is “satisfactory,” 
particularly in such a way that the public will accept the 
demonstration as proof of the “practicality” of the examination. 
Faced with this dilemma, the majority of civil service agencies 


TEST CONSTRUCTION 147 

formed in a setting that emphasizes speed and competition. 
Theta in turn, they say, create anxiety and thus poor perform- 
ance. The public subscribes to the belief that “accuracy is 
more important than speed" without recognizing the corollaries 
that for many jobs, rapid it preferable to slow accuracy, and 
that speed and accuracy,' at least in relatively simple talks, 
tend to be posmvely correlated 

Many candidates, and particularly those who have failed 
or attained low ratings on a time-limit test, share the layman's 
preference for work-limit tests Some may believe that speed 
tests unduly penalize the older, more experienced worker The 
fallacy in this argument u twofold Ant, although speed of per- 
formance of tome functions, as measured by tests, probably 
does decrease somewhat with age, in reality tbe speed of the 
performance of the soma candidate at different ages (within 
the usual age range of candidates but excluding the age of 
senility) does not usually differ markedly; and, second, if the 
speed of performance of particular functions did decrease ap- 
preciably with age, then, for those positions for which speed of 
performance of these functions is important, reflection of this 
decrease in the teat score would increase the validity of predic- 
tion. Actually, comparisons referred to by candidates are usu- 
ally not based upon the performance of the same candidates at 
different age levels but rather upon that of persons who differ 
not only in age but also m basic abilities and who still would 
differ in abilities even if they were of tho same ago The older 
candidates ihfty get lower eeores on a speed test and thus appear 
to be penalized The score is not a result of age hut or indica- 
tion of leaser ability., Reflect that most older candidates for a 
job at a level for which a speed test Is frequently given have 
worked for a number of yean without progressing as Jar as jobs 
which will be beginning jobs for the younger candidates Thus, 
among candidates for clerical positions, those aged 55 do not 
represent the same kind of sample of 55-year-olds as those aged 
19 represent of 19-year-olds, 

Regardless or the false premises underlying this predilection 
for power teats, probably the beat forecasts of performance in 
the higher-level position* will result from tests mch time limits 
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BO liberal that only habitual laggards will have difficulty in at- 
tempting all the items within the tune allotted, For the major- 
ity of clerical positions, however, where speed of performance 
clearly is an important component of job proficiency, achieve- 
ment of the best possible prediction now will m the long run 
outweigh any advantage that might be gamed by a concession 
to public opinion at the moment The inclusion of a speed teat 
as one component of an examination, even for a clerical class, 
may wisely he accompanied by a campaign to educate the In- 
terested public on the reasons for its use. 

Length. Every person who has ever attended school baa 
opinions about examinations Thus public opinion impinges 
on die proper length of a teat. The school examination, taking 
from 30 to 50 minutes, is regarded as typifying the most trying 
ordeal to which a candidate should be subjected It is forgot- 
ten that normally the results of any one such school examina- 
tion sro considered along with the results of several others, < 
daily observation, and the appraisal of performance on many 
additional assignments, and, more important, that such all ex- 
amination u designed to measure achievement in a relatively 
limited area of a single subject-matter field. In contrast, the 
civil service written test la supplemented, if at all, only by a ( 
rating of education and experience and possibly by a brief oral 
interview or a performance test; and it is designed to sample a 
much greater complexity of knowledges and abilities 

When candidates dearly realize this distinction, when they 
Understand why a tut that adequately samples the major fields 
ml. ig to a job is more reliable and more valid than one re- 
stricted ta coverage, they prefer tests sufficiently long to yield 
a reasonably just appraisal of their job potentialities Al- 
though candidates may complain (hat long examinations are 
endurance contests, experimental work on prolonged mental 
effort indicates that the affect is more potent than the effect. 
Unfortunately, some civil service jurisdictions capitulate to 
public pressures for short examinations instead of enlightening 
the public on the virtues of comprehensive sampling 

Type of Item Decision on the type of item best adapted 
to civil service tests should be weighed against adequacy of 


ISO EDUCATIONAL AND PSYCHOLOGICAL MEASUIHUKNT 

all time I urut, simplifies test administration and hence may 
contribute positively to test validity. Thu is one way of over- 
coming the handicap of monitors poorly trained or too unstable 
to moot emergencies with poise and skill. 

Repealed l he of Items In the educational world test 
questions are sometimes used more than once, to the gratifica- 
tion of certain fraternities that keep files of questions available 
Scores tend to be higher on the second administration of a test 
Perhaps in college repeated use is not very serious, since a courae 
marie usually does not depend on one examination alone and 
since a mark in a single course may be of no great moment. 
Institutions that are placing dependence on comprehensive 
examinations and little if any an course marks, however, are 
increasingly 'following t >,e practice of using test questions only 
once. Hie problem or maintaining the confidential nature of 
examination materials is even more urgent for civil service 
jurisdictions There may he organized efforts on the part of 
"cram schools” to obtain access to examination items used, 
some candidates may apply for the sole purpose of memorizing 
assigned portions of an examination In die absence of such 
studied attempts to sabotage a ment system, candidates never- 
theless may remember a few items ui detail and another group 
q( questions to “look up.” Whether or not this factor would 
give any odds ta these candidates at their friends if rite same 
items were repeated, some members of the public might think 
there would be an advantage. Thus the confidential nature 
of the examination fuses with the question of public relations. 
A civil semce agency better maintains the support of the pub- 
bc if it can grve assurance that no appreciable advantage could 
accrue to any candidate became of previous me of items. 

Statistical Andyjet of Items. From the point of view of 
the reliabpity and validity of a test, inclusion of a few poor 
hems is negligible if the test as a whole is long enough. In 
one way, however, the problem created by indefensible items is 
greater in civil service tests than m almost any other kind 
This is time because a difference of one point may affect a can- 
didates rank order and alio may determine his passing or fail- 
ing. Kenre he may or may not get ajob. Since a difference 
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coverage of the pertinent areas of knowledge and abilities, 
objectivity of scoring, and ease of administration. All things 
considered, the use of a large number of objective items is In 
general preferable to reliance on an essay examination. In 
civil service, objective teats have largely superseded essays In 
agencies chat still use them, the tendency towards supplementa- 
tion by the objective type increases 

If broad, general essay questions are uped- m an effort to 
cover particular fields of subject matter, different candidates 
may not actually be answering the same question, a factor that 
may nullify attempts to place them in an order of ment If, on 
the other hand, essay questions are more pointed and limited 
in scope, the coverage or the requirements is automatically 
restricted, although more reliable grading can be achieved 
The objective test not only has the advantage of permitting a 
broader sampling of pertinent knowledges and abilities than the 
essay test but also, if properly used, will almost certainly lead 
to markedly greater reliability in scoring. These are highly 
important advantages in civil service, where areas of subject 
matter to be covered are broad and where unreliability of scor- 
ing may have a greater effect upon human destinies than m 
almost any other field of testing, 

Test scoring should be objective and also as simple as pos- 
sible in the interests of efficiency. Wherever feasible, the same 
kind of item should be used throughout and the same weight 
assigned to eaeh item. The multiple-choice form lends itself 
admirably to most purposes. Many forms that appear to differ 
are simply variants of this form. The argument that use of a 
variety of forms adds interest to a test is insignificant, aince the 
competitive setting supplies more than enough motivation 
Arousing interest in a teat per se is an empty gesture. Items, 
and even the several response# to an item, are sometimes dif- 
ferentially weighted in civil service testa, but there » growing 
awareness that unweighted and weighted scores for a large num- 
ber of items correlate so highly thit little is to be gained by 
differential weighting. 

Use of items of a single, readily understood type, accom- 
panied by dear instructions to candidates and with one over- 
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of even one point hat this importance, great care should be 
exercised to exclude from a test items that might have a nega- 
tive validity The problem Is of consequence, too, as it relates 
to public fsith in the merit system A widely heralded appeal 
based on a few weak hems, which may really be of no great 
moment insofar as they affect the over-all reliability and valid- 
ity of measurement, can go a long way toward undermining 
public support of the ment principle. 

In speaking of the validity of a civil service test, we should 
have in mind the extent to which the test serves its basic pur- 
pose, prediction of performance on the job To estabbsh incon- 
trovertibly that a test has validity for this puipoie is of course 
difficult. II a candidate population is used, the failing Candi- 
da tea do not get Jobs, so that there is no information regarding 
their work, and the distribution of job performance indices is 
thus curtailed to an indeterminate extent. Even more trouble- 
some is the notonoui unreliability of service ratings commonly 
used to appraise success on the job Made by a number of 
raters, with individual standards, on employees performing for 
varying periods on jobs that differ although grouped under one 
classification, service ratings are sensitive to many uncontrolled 
and uncontrollable factors Too frequently clearly recogniz- 
able differences in job performance are obscured by a tendency 
to rate employees in the "above average” categories, with the 
result that distributions of ratings exhibit merited negative 
skewness And this is only one of the frailties to which the 
rater is heir! Unfortunately it is almost impossible to sure 
just how unreliable service ratings are, since conditions for s 
realty crucial experiment in this area have not been met in prac- 
tice. Probably, however, .30 to .50 represents a oafei estimate 
of the Pearson correlation coefficient between two complete iy 
independent sets of ratings, made by raters assumed equally 
familiar with the work of employees on jobs withm the same 
class, using rating forma developed with an ordinary amount of 
skill and care A similar correlation of ratings made specifi- 
cally as a criterion against which to judge a teat would rend 
to be higher. In any case, problems ,ef securing as a criterion 
reliable evaluations of job performance on a population of Wty 
ficient size are very great and In most instances insurmountable 
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Establishment of test validity /or a particular group 0 f can- 
didate* ii troublesome enough, but efforts to prevalidate on one 
poup a te*t ro be used on another group for selective purposes 
are mon hazardous still 

If a population of employees rather than candidates is used 
the problem of proportionate representation of those who would 
fad the test If they were candidates remains Assurance is 
lacking that an employee group, for purposes of a validity ex- 
periment, is representative even of the passers among a candi- 
date group Parts of a test that are satisfactorily valid for 
a candidate group may not differentiate among employees, who 
have had experience on the job and who may all have learned 
to do certain kinds of tasks that only the abler of the candi- 
dates could perform Moreover, if the confidential nature of 
tests Is to be maintained, an agency may not wish to be in the 
pottaon of repeating, either wholly or in part, a test from which 
then* may have been "leakage." 

Attempts to secure an experimental grdup of subjects who 
ire neither candidate* nor employees face these difficulties as 
welt as the impossibility of obtaining any measure of job per- 
formance. * 

Confronted with these obstacles, the experimenter in the 
field of civil service teats usually resorts to a candidate popula- 
tion and an internal criterion, which is customarily the total 
■core on a test composed of items sampling a variety of areas 
of knowledge and abilities Conclusions should be drawn only 
charily from analyses based on BUch a multifactored criterion. 
Since different areas are represented in the va nance of the total 
scores to differing and usually unknown extents, dependent 
upon the variances and interrelations of the part scores, gen- 
eralisations on relative validities are apt to be inappropriate 
and misleading. The item-test coefficient must bo interpreted 
m relation to the item difficulty for the population in question. 
An item that fails to differentiate among candidates for one type 
of position may discriminate positively for another Any items 
with negative coefficients should be carefully scrutinized with a 
view to improving the item if its re-use is contemplated and, in 
any event, to gaining insight into the charactjnstlca that may * 
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iMd to negative coefficient. An item that coml.tes nega- 
■mjjFwith u “Iterion may, however, be positively 

related to the ultimate criterion to be predicted. 

For mtilnplMhoice iteme, it la advantage to examine 
ImV r ™<5tient, which may be regarded a, a 

of summary atatunc, but alao tha relationship of each 
ption to the criterion Thie ha. beep approached in virioue 
weye-by correlating each choice with the enrenon, by Bnd.. . 
the mean criterion score of those who select ecch choice, by 
finding the proportion of pereons in each of two controlled 
enttnon groups for each choice, or by come other method, de- 
pending upon the type of Item index preferred With access so 
rtuehnd of information or each choice, the examiner can often 
prove item, and learn how to ctmetmce better onet Within 
certain limitl, which of the many variant fauna of item-analyiia 
tachmquu one uses .name relatively unimportant. Ideally, ,t 
may be preferable to compute for each choice for eech item a 
omletion coefficient (either a bi-nal, a pornt blaerial, or 
both) with a multivariate criterion* If mtenuu cent.™ in 
only a imull number of items, tbi. approach i. entirely prac 

rc ' Ul " " v " nl1 fiunrireda or oven 
riioueand, of irema are available, a degree of .mtleueal refine- 
ment may profitably he eacrificed ,n ,h. mtereaia of oarng a 
,° f d "'“ “ hu,d ' particularly if feeibnea fur 

mre .T 'T* 4 In rb “ “«■ > "aLaUy m„f«- 
tory technique may be to uae the tetiachonc correlation coeffi- 

™ th 1 ‘, h K ™T n fitchotomized at the median. Here, 
S 1 . 1 ' W,nIb * of t, !“' to find ui addition the proportion, of 
candidate, above and below che enrenon median who .elect 
each choice For most poiposee, a practical index of item d.f- 
fituky i> amply the percentage of candidate, who relent the 
dost answer 

Some agencies that do not at the present tune want to re- 
P*at ite ms nevertheless apply item analysis techniques with a 
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view to gaining insight into the characteristics of good and 
poor items, Both subject-matter consultants and test techni- 
cians will profit from reviewing results of statistical analyses, 
even though the values may be in large part indirect 

As data accumulate over a period of years, it should be 
feasible, at least for classes attracting large numbers of candi- 
dates, to construct an examination largely from pretested items 
only a small number of which have been used before ui any 
one examination. The results of statistical item analysis 
should not be applied blindly, for both the reliability and valid- 
ity of items may be altered strikingly by changes in the social 
milieu. A test item valid at one time may be based on a con- 
cept that later becomes such common knowledge that even the 
poorest candidate answers it correctly Or it may hinge on a 
concept that becomes outmoded For such reasons as these, 
even were statistical indices available for a group of item* from 
so large a number of different testa that the confidential nature 
of examinations would not be jeopardized by judicious repeti- 
tion, competent consultants 1 review or items shortly before their 
inclusion m a test would still be essential to sound test con- 
struction. 

Perhaps m the not too distant future factor analyses of civil 
service tests Can dispel the problem created by the unknown 
contributions of several factors to tho variance of internal cri- 
terion scores Then items can be correlated with separate fac- 
tors instead of with a hodgepodge. Even so, the question of the 
appropriate weight for each factor in the composite could not 
be answered rigorously. The solution to this problem awaits 
development nf a satisfactory external entenon 

Etiabhihmtnt oj Critical Scares As those expenenced in 
test construction know, even when statistical data are available 
the difficulty of an item cannot usually be predicted with high 
accuracy unless one has worked a great deal with the particular 
type of item and » predicting the difficulty for a group of can- 
didates having a known level of ability on the type of item In 
question Although tho prediction for each item may not be 
accurate, many civil service agencies assemble groups of items 
with a view to setting the passing point on an a pnari hails, 
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In some cases, the law or rules under which the agency operates 
mako such a judgment mandatory In many others, where a 
fixed passing point is not so specified, the agency nevertheless 
considers the predetermination of passing points desirable, pri- 
marily from the point of view of public acceptance of the 70% 
passing point via the educational system Truce with this 
concept, hoary with tradition chough it is, is not entirely with- 
out compensation. 

Although the judgment of the appropriate difficulty or a 
test is admittedly hard to make, adherence to a fixed percentage 
of the total number of items as the passing point, where its use 
is not clearly inappropriate to needs of operating agencies or 
unjust to candidates, puts the examining agency in a position 
to combat pressures for setting passing points so that particular 
candidates are passed. Thus the agency is better able to be 
impartial Not infrequently, however, even in agencies (hat 
have a fixed percentage passing point set by law, scores , ate 
transmuted if an examination appears to have been so difficult 
that a register resulting from it would contain insufficient 
names to fill vacancies Usually a linear transformation is ap- 
plied so that some percentage score less than 70 becomes a 
derived score of 70 and, obviously, so that some ocher condition 
is satisfied, such as the original top score being equated to a 
score of, say, 95 on the derived scale The most appropriate 
second condition vanes with the character of the original dis- 
tribution and with the desired properties of the distnbution 
of derived scores 

Although it may be considered advantageous to attempt to 
estimate test difficulty so accurately that no transformation 
of scores will be necessary, transmuting upward is considered 
preferable to transmuting downward, because candidates, while 
welcoming acorn seemingly higher thin actually attained, are 
reluctant to accept scores lower than the percentages of items 
answered correctly Moreover, a test that is difficult will dis- 
criminate among candidates better than one that is too easy 

Several factors m addition to the inherent complexities of 
estimating test difficulty magnify the problem for the civil 
service examiner. One » that the nature of the candidate 
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population vanes with the labor market) and the extent of 
chingft. as it pertains to the abilities and knowledges being 
measured, is not easy to forecast. It vanes with recruitment, 
again tinpredietably- It vanes with changes in minimum 
qualifications and aalaiy level, in such a way as to be especially 
bothersome if minimum qualifications are lowered at the Bame 
time that the w Ury level is increased, a combination not un- 
usual In a tight labor market In fact, with the multiplicity of 
factor, operative, it ia aurprUiog that results as satisfactory 
u those commonly achieved are possible. 

Sometimes effort is made to set the critical ecore at a 
"hreak" In the distribution, on the grounds that failing candi- 
dates will accept their bt more readily. Some evidence to 
bolster this point of visw probably could be adduced At 
times, however, the procedure seems to be baaed on the assump- 
tion that peculiar significance attaches to a segment of the 
rango within which no scores fall, This u nonsense. It it 
doubtless possible to construct a teat having scores with a 
bunodal distribution Whatever gaps appear in distributions 
of civil service test scores, however, are due to chance, not to 
rational design If the number of candidates is sizeable, there 
will be no appreciable breaks In the proximity of any reasonable 
critical score, and no matter ^hera this scare ii set there will be 
scores just below it. All things considered, an agency should 
face this fact tough-mindedly and not waste time in seeking 
breaks m score distribution* or in creating them artificially 
Compiling Related $teammUons, A task almost peculiar 
to civil service examination construction it that of aisembluig 
for a single administration examinations for sometimes as many 
as 35 or 40 classes, but perhaps more typically Tor from five to 
15 If there are no common requirements of knowledges or 
abilities among classes for which examinations are to be held, 
then the examinations are aimply assembled independently. 
If there are somo requirements common to two or more classes 
the examinations ordinarily include some common items sam- 
pling the overlapping ones, particularly identical degrees of a 
given kind of knowledge or ability 

One argument for using overlapping items for related classes 
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ably the mmimut n number of differentiating items for two 
anamination* for closely related desire, eacb cons is ting of 200 
items, ihould be In the neighborhood of 50 to 70 items. The 
optimum' number differs considerably from one* pair of classes 
to another. 

Order of /rents for Related Enaminatlonr, A special prob- 
lem of ordering Items ames when, say, 15 examinations are 
encompassed in a booklet of, say, 1000 item*, many of which are 
common to two ot more daises, If they »rc ordered strictly 
according to subject matter, the items in any one examination 
are scattered, hence candidates who have to skip several groups 
complain that the mechanic* are confusing and may waste their 
tune. On tho other hand, If the items are ordered in such a 
way ax to minimize the number of ''breaks*’ for the examina- 
tions having the largeit numben of candidates, the items can 
no longer be ordered entirely by subject matter, nor can an 
ordering from easier to more difficult be strictly maintained. 

If administrative controls are adequate, perhaps the best 
solution is to assemble for each candidate a booklet containing 
only the item* pertauauig V the examination or group of ex- 
aminations he u talqng, Such a plan creates limitless con- 
fusion if the program ia improperly administered. 

A solution that' places a leaser burden on sound administra- 
tive control* it to retain the plan of assembling related ex- 
aminations in ohe booklet and to compromise among thp 
objectives of arranging Item* (1) In the order that minimizes 
breaks m each examination, (2) in the order that seems logical 
so far m subject matter is concerned, and (3) in order of dif- 
i ficulty. Perhaps the simplest way to arrive at an ordering 
reasonably sxtiriaetory from these three points of view ja first 
to establish an order that. mm un not breaks for the mast "pop- 
ulous” daises, then to adjust the ordering by subject matter, 
then further to rearrange the items ^o that for any single class 
the more difficult items in any distinguishable area follow the 
easier items. 

Selection and Training o / Subject-Matter Consultants, 
emphasis wu given to subject-matter consultation m 
dvM rervfce test eotiitrucffon. Brief mention will now b« made 


ia that some candidates may want to take more than one 
examination, if the extent of overlapping is slight, such candi- 
date* may not complete some of the examinations m the time 
allowed To permit candidates to take several examinations 
for related classes in a single day (usually the maximum tune 
Tor administration of an examination program), the only alter- 
native to overlapping is shortening the total length of each 
examination, which is of course disadvantogeou* from the point 
of new of reliability Same jurisdictions etc in the other di- 
rection by constructing tests with so large n proportion of over- 
lap that differentiation among the tests » unreliable The 
effect of this error is that candidates who fad ona examination 
are likely to para another for a higher class in the same sene*, 
because of rhe chance element m the small number of differ- 
entiating items. 

Such a result Is not the reduetto ad abjurdwtn of the ex- 
amination process that it may seem at first glance. As a mat- 
ter of fact, identical examinations for classes that differ m 
degree rather than in kind would m some instances bo ap- 
propriate were it not for interpreting to the public several 
passing point standards On the other hafid, the same passing 
points might be set for the written component of the examina- 
tions for several classes in a hierarchy, if the total examination* 
for the classes were differentiated on some other basis Such 
a practice has sometimes been followed by the United State* 
Crvd Service Commission. For jurisdictions more limited in 
size of the public served, different written examinations for 
separate but related claesee seem preferable, because some 
the candidates for each are likely to be placed in the same 
agency on the basis of the examinations. 

Roughly stated, the most useful principle for differentiating 
examinations for classes In a series Is that the examinations 
should contain enough different items to reflect reliably the dis- 
similarities in requirements and enough common items to en- 
able candidates who so desire to take three or four examination! 
at one sitting A further important advantage in the use of 
common items » the economy effected in item construction and 
review. Although there Is no mle applicable to all cases, prob- 
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or some of the problems of using consultants in this capacity* 
Both selecting and training consultants are simplified for those 
examining agencies needing full-time consultants and having 
money to pay for them More typically agencies seek inten- 
sive but intermittent service*. 

Unfortunately those who ire recognized a* authorities in 
their fields are also those who are likely to be employed in posi- 
tions from which release for temporary assignment elsewhere 
is not easy to obtain* On the other hand, even if an examining 
agency has a continuing job of such magnitude os to warrant a 
permanent assignment of a consultant, many of those who 
would be acceptable as consultants are loathe to abandon front- 
line operations to undertake the cask of predicting the per- 
formance of others They hesitate to leave a position that thsy 
know and like to pioneer in a job that they may find difficult 
to understand and that may offer no clear line of advancement 
Having taken this hurdle, they become dissatisfied if oppor- 
tunities (or contact! with others m tlieir field we too liuund 
or if for some other reason they "go stale” on the job. Because 
of these (acton, some agencies tend to prefer making a series 
of temporary appointment^ possibly on n part-time basis. 

Probably no single solution to the problem of velectmg and 
training subject-matter consultants would meet all needs. 
Where it is appropriate and feasible, however, s plan of having 
a senior consultant <on a full-rime and permanent basis and ad- 
ditional consultants on a part-time or temporary basis has the 
advantage of providing continuity to the examination program 
while at the came time securing a reflection of different points 
of view in the examination content. This plan, also simplifies 
the training problem, because a permanent subject-matter con- ( 
suit ant who understands examination construction can inter- 
pret aspect* of tins field to other consultants in theepmo area, s 
often more efficiently than can the poyehoftietriclan 

Some persons who are sought as consultant* and who may 
be recognised ms authorities simply carnet be taught to con- 
struct examination*, Sometime* the difficulty Is pppareidy 
attributable to temperament, sometimes to pattern of abilities, 
mote often to t|n combined influence of both factors, To do* ( 
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■ jjjjt fl consul cant cannot be taught to construct items may 
rtouwe only half an hour end again, for one who approaches 
Stask with' interest and enthusiasm, may take a month or 
Zrt Such Ineptitude w not peculiar to any held It can he 
m psychology, art, grammar, accounting, social work, 
hw music, and statistics The Fruitful solution to this problem 
lf to «t another consultant as tactfully as possihk 

Once a consultant is found who is both interested in and 
gdept ar jtcm construction and test compdaoon, he should he 

vm, interpretation on the desirable length of an examination, 
scoring procedures, time limits, reliability and valid, ty, trans- 
mutation of scows, weighting of several components, and simi- 
lar concepts. Two of tho most common misconceptions of 
Hibjcct-mattcr consultants are confusion between the difficulty 
and the validity of an item and failure to differentiate between 
a teat as a predictive instrument and as a teaching device. 
Much attention must be given to overcoming these and similar 
fallacies related to examinations , 

Magnitude of the Tmk The final problem of which brief 
mention will be made is the scope of the examining job to he 
fauwithm budgetary limitation b. Although monies allocated 
ta the examining function in any area are rarely of staggering 
proportions, probably the public personnel agencies suffer the 
molt from paucity of funds The number and variety of jobs 
for which examinations arc to be constructed and the sizes of 
the populations to be examined in normal times are in many in- 
unices almost unbelievable. In a single jurisdiction hundreds 
of 1 it« may be affected by a single examination program Yet 
the annual budget for the examining function probably would 
represent only * small fraction qf the sum allocated yearly to 
dM "supervision of reindeer in Alaska." Despite the handicaps 
of nadequtte staff and limited facilities for research, the notable 
progress of the last decade places, on those in the field of 
measurement an obligation for continued effort toward the 
rolution of the critical problems that remain 


RELATIONSHIP BETWEEN KUHLMANN-ANDERSON 
INTELLIGENCE TESTS IN GRADE 1. AND ACA- 
DEMIC ACHIEVEMENT IN GRADES 3 AND 4 • 

MILDRED M. ALLEN 

New Rocbdc Public School i, New Rochelle, Now York 

Tuu study u part of a doctoral research on the prediction 
of academic success of elementary school pupils by means of 
the Kuhlmann-Anderton Intelligence Tens. The purpose of 
(his study was to determine the predictive value of the KvJU~ 
mamt-Andersoti Intelligence Tens as o wholo when adminis- 
tered In Grade 1, In the fields of reading, arithmetic, and spell- 
ing in Grades 3 and 4 as measured by the New Stanford 
Achievement Test, 

Data 

The subjects for this study were three hundred and twenty- 
seven pupils from ten elementary schools in New Rochelle, 
New York, Complete test results of these pupils from the 
school yean 1936-37 to 1939-40 were used, Tire tests were 
the Kuhhnann-Anderson Intelligence Test and the New Start- 
ler d Achievement Test 

Procedure 

An alphabetical class list of fourth-grade pupils (1939-40) 
by schools was obtained from the school census cleric, From 
■'this lilt pupils were selected who were originally in the fine 
grade hi 1936-37. A checking and ie-checkmg of all test re- 
sults dating from Grade 1, 1936-37, and including the fourth 
grade of 1939-40 was made ur order to select a group of pupils 
who had taken the complete battery of tests as used in the 
present sfudy 

* Parr d ■ ttudr for a Doctor 1 ! dbrennon cooplttcti at Naw York Unifovlty, 
Gnfcm* SAod of E dotation, 1940. 

Ml 
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The KiMmann-Anderson Intelligence Test lor Grade J 
(Second Semester) ,v/u administered in February, 1937, when 
the pupils wore midway through the first grade. The New 
Stanford Achievement Test (Primary Examination) Form Z, 
was administered in April, 1939, near the close of the third 
grade, and the Advanced Examination, Form W, of the Bame 
lest vu administered to the same pupils in the fourth grade 

m October, 1939. 

The Kvhlmann-Anderson Intelligence Test was personally 
administered, scored, and re-scored by the writer The New 
Stanford Achievement Test (Primary Examination) was ad- 
ministered In the elementary schools by the principal or a 
teacher who had test experience and training. Both principal 
jud teacher-examiner received Instructions for the administra- 
tion, scoring, and tabulation of test results from the writer 
The tests were scored* and double-checked by specified teach- 
ers ui the respective schools, by the principal, or by the teacher- 
examiner. The Nea Stanford Achievement Test (Advanced 
Examination ) was administered by the writer to all fourth 
grades in October, 1939 This test was scored and re-ecored 
by an assistant under the direct supervision of the writer, 
Rsxidfj and Interpretations 

The mental ability of the pupils used in this study is shown 
in Table 1 Correlation coefficients between measures ob- 
tained from the Kuhlmann-Anderson Intelligence Test in 
Grade 1, and performance on the New Stanford Achievement 
Tests in Grades 3 and 4 are shown ui Tables 2 and 3 


TABLE J 

JfMv, 1*4 JtauM-a/ IjQ't m lb Iohlm»wvAj*dtr*Jn IrtUfljMoc* T«t 

ta Crete 1 mi H Qni*4 


T« 

Muo I.Q. 


JUnto 

fcAlamw Aatairoa 

Tm-Cud* 1, Fta, 1937 

.. 100.7 


£3-125 

ZHUnsno-AadafU 

Tai-Gnd* 4, fkt, 1939 

.. . »J 

ll» 

’ 67-132 


The mean I.Q ’s of the pupils indicate average ability for the 
group as a whole, with the range of I Q.*s showing the wide 


scatter of ability commonly found in heterogeneous grouping. 
The intelligence level of the pupils remains about the same in 
Grade 4. (These were the same pupils tested in Grade 1 ) 


TABLE: 

CmUmIi of ComUUon A si -r n KnMmua-Aadi 
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•nci ta Gmis I aai Ik N«w Blsnfonl AtUunnt Ten (Pnmuy 
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won mwiui| . . • . 
Rndhi Averts* ... 
Anlhnnle Rnsaning 


Amhmrtc Comma 
Anthsittie Arenjp 

SSU* " 

Jagg! rffl.i;.;. 

tUoilmM UlMUH 


* Tta PtAv, a# Pa Ctnt of Ava«t» Dontapmm, ■ n Ms Atnad ta 
divxlinf in buHvriiaTi mental iimt points by the araafs moral vnlc pofon for hn 
sio aroun, ituaral units bone ibromlncd by raimaim u ■ poise tola doisnsd by 
Httait Tht Pc-Ar b entanf by Kublmtno in tbs L(L tun it b nun naun 
for ntaa mr a p*m>o ad ywa. 


Table 2 reveals coefficients of correlation between the Kuhl- 
monn-Ander'jon measures and educational achievement rang-* 
ing from -32‘ to .53, with twenty of the thirty coefficients bc- 
t tween 40 and -SO. In view of the fact that these coefficients 
have standard errors of from .03 to -OS, no significant difference* 
among the coefficients are indicated. The 1 Q, Pc,Av., (Per 
Cent of Average Development) and E-Q, all include one com- 
mon element, namely, the chronological age (CA), which Is 
not included in die mental age (MA) score It may be noted 
that some persons regard as spurious correlations of ratios in- 
volving the same variable denominators* The critical ratio 
for tho difference* of the coefficient* of .53 (I Q. or Pc_Av,, and 
EA) and .67 (I Q. prPcAv , and E.Q ) is 3.1. 

Whether one considers the MA, I.Q., or Pc,Av seems to 
make little difference since all yield substantially the same eo- 
effident * with the various subteit results on the Stanford Pri- 
vjTp Gdfo*, hyttowlifc MsMi, p 
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binary Aehwsment Test. One part of the Stanford Primary 
Achievement Test a predicted sa well from the varioua Kukl- 
monn-Anderson measures as any other part. The only variable 
derived from the Stanford Achievement Test which shows any 
significant differences m the coefficients is the E.Q. An exami- 
nation of the lost row of Table 2 shows that the correlation 
between M.A. and E Q w about the *ame as the other coeffi- 
cients in the table The relationships between both the EQ 
and I.Q., and E.Q, and Pc.Av , are substantially higher ( 40 
compared with 67 and 67) The critical ratio of the differ- 
ence between the correlation of M A and E.Q, and the cor- 
relation of I.Q. and E.Q. is 5 4 Thu indicates undeniable 
statistical significance of the difference, 

Excluding the coefficients involving E.Q., the median coef- 
ficient for Table 2 u 44 While this indicates some degree of 
relationship between thtKuhlmann-Anderson scores and edu- 
cational achievement, it is by no means great enough to be of 
much value in predicting third-grade performance from the 
Kv/Uvumn-Anderson- Intelligence Tests administered in Grade 
1. Due to differences in content material of the earlier KvhL 
mann-Andmon Test (non-verbal) in Grade 1, and tho verbal 
content of the Stanford Primary Achievement Test in Grade 3, 
It appears that long-range predictions of atadomic achiqyement 
would not be reliable To insure a greater degree of reliability, 
intelligence teau should be repeated annually, and preferably 
within the grade for which predictions are made, The coeffi- 
cient of alienation corresponding to an r of ,44 is .8980, an in- 
dication that errors of prediction would bo reduced by only 
102 per cent by the use of tho Kvklmonn-Andsrson Intelli- 
gence Test in Grade 1, instead of making predictions without 
the tests. The highest coefficient of Table 2, .53, between I Q. 
(or Pc Av ) and EA- yields a h (coefficient of alienation) 1 of 
.8542, indicating a reduction in errors of prediction of only 14 68 
per cent better than chance 

The most predictable measure obtained from the Seta Stan- 
ford Achievement Test, Primary Emamtrtation, when predic- 
tions are made from the Kuhlmnn-Anderson Intelligence 

■J P GaDfard, Pijdumetda Mtiheds. p 362 
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Test administered in Grade 1, eeems to be the Educational 
Quotient (E.Q ) The r of 67 between I.Q (or Pc Av.) and 
E Q. yields a k (coefficient of alienation) of 7424 and indicate* 
a reduction m error of prediction of 25 76 per cent better than 
chance The E.Q , however, does not give any indication of the 
actual level of achievement, but only of achievement compared 
with C A. It is significant, however, os an index of educational 
b lightness In thia study the PcJkv (Per cent of Average 
Development) doe* not appear to have any advantage (for 
predictive purposes) over the IQ or MA scores obtained 
from the same test 


CMffiriMU o} Condition Btimn Knhlmirm Anderwm Intdllwnce Tm 

cue U Grab J ni Hew Stirfanl Juhknmmt ^ 

Enmiiutvui, Form W) Pmrfomaau is Grab 4 


Snckttd Achievement Te»t — QratU 4 


Kuhlmann-AnAenofl Ten— Or.^. J 


MA 


IQ. 


PcAt 


Pangnpii 1 
WflM Mowing 
Rejdini Avnp , 
Anthmiue ScmobId* 
Anlhromc Computidon 
AnthirtrtK Annp . 

Si Aan w " ‘> .. 
EduaiiMBl A|« 
Educuonal Quotient 


J7 

JO 

JS 


4 


Tabic 3 is very similar to Table 2, except that in this case 
the relationships shown are those between the Knhlmann- 
Anderson Intelligence Test for Grade 1, and achievement mea- 
sured at the beginning of Grade 4 The Advanced Examina- 
tion, Form W of the New Stanford Achievement Test was used 
in this instance. The intelligence test scores are the same but 
the achievement test scores were obtained approximately six 
months later (after a summer vacation) and were obtained 
, from a somewhat more advanced examination 

None of the coefficients in Table 3 are significantly different 
from the corresponding coefficients of Table 2. The general- 
izarions regarding Table 3 are the same as those drawn from 
Table 2 Excluding those coefficients involving EQ,, the 
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range of the coefficients of correlation is from 30 to £6 X with a 
median r of 48, which u higher by only one a, of .(H than the 
median r of ,44 m Table 2 In this instance it is again evident 
that the EQ » the moat predictable measure of educational 
achievement. The reduction m enon of prediction due to the 
magnitude of r is practically identical with the cant* ponding 
reductions in Grade 3. The M A , I Q., and PcAv are about 
equally effective for prediction There seems to be no superior- 
ity of the PcAv aver I.Q as a predictive measure 

A study of Table 3 show* a high nr correlation between all 
indices of intelligence (MA, I.Q., and P&Av.) and arithmetic 
reasoning, than between the tame indices of intelligence and any 
part of the reading teau. The correlation between the indices 
of Intelligence and arithmetic avenge is higher than the corre- 
lations between the came indices of intelligence and reading 
average. The mosr noticeable shift in prediction is that of the 
arithmetic scores 

Summary 

The most predictable meature obtained from the New Stan- 
ford Achievement Test, Primary Examination (Grade 3 ) and 
from the Advanced Examination (Grade 4) when predictions 
are made from the Suhlmann-Anderson Intelligence Test for 
Grade 1, seems ta be the EQ. (r«= £7). Since E.Q, does Hot 
give any indication of the actual level of achievement, but only 
of achievement compared with C-A, thn is significant only at 
an iodex of educational brightness ThePcAv In thi* instance 
docs not seem to have any advantage (for predictive purpose*} 
over the I.Q. or MA scores obtained from the Kuhlmarm- 
Andenon Intelligence Test. All ihrte indues of intelligence 
f MA., I Q-, and PcAv ) are about equally effective for preduj- 
tioo. 

Coefficients of correlation between the Kvhhmnn-AndefJon 
measures in Grade 1 and educational achievement in Grade 3 
range front ,32 to .53, and between the same test (KvJdmann- 
Andertan InttUigtne* Tat, Grade J) and educational achieve, 
ment in Qrado 4 from JQ to ,56. These low correlations indi- 
cate that long-range predictions of educational achievement 
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based on only one group intelligence test in the first grade are 
highly questionable, 
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tut itthi J31 I*vri tor ATr (bore 30. Head for cautim In evaluation toe signifi- 
cance of codEcciata ohnioad from data involving tic ranbnga u Indira red, The 
etriria arotVain with uMii r uuend aawu aa in toriee if mttoom of de t e rminin g toe 
aig i n fao c e of tank dWenace cosSnare. (Coum»y PsyckometrikaJ) 

Tinker, Mfiaa A "Spod, Power, ud Level in tha Revued Mlnnmoa Papa Form 
Board Tret." Journal of Gtnelk AyeAefujh UCIV (1944). »3-97 
The Revised Mimesoi* Paper Form Board Tert ni adnunuiered tn 10) enh 
Vanlry aophorunra to analyze to* criiiloruMp of iha work lorot method id mruma- 



172 ^EDUCATIONAL AND PSYCHOLOGICAL MRASUIBMENT 


mat rnuiun* w ‘‘ipood’ 1 *o»W to rt» uwimiw nra* hish»o re^un* ■ tw- 
KotB ind ihi rdaHcn cf both d lhae to innnra rise ®fw» ratonc* m pwtr 
im 0p «d ind Iwd iron a win found to ran mfop*pd#ntly A m»wr prooor- 
dst of ihi DOW# KQR wu icaMinted lot h? iw«i ind IctiI, Wjin ipcri CMlWjf* 

Wre fdinray more to dw power law thin Icvd. Th* i»dy IndifltM onto i dtont 
tanhifan berwren uneQigmt ind the RnOU Ptftr form board TuL CtlM— 
Am UtH thy 

ffhoiT, Robert J end G.Wari 1 Rjdurt H- "Tbo Ootfft afTtfi ml Itaa Ut 
ihdltT m Rtlitfaa u Victor Pimm." PjycAmwi/UA VITI (1913), 247-264 
It U ihowo thii ippraicha other ihia too laKfu] eoralitonry method ol 
Mtmitinc un rdlibihS ire mhtr In unfunny tn Ind id toe nuns tcrunl 
muIil Tht eomraoidy Madam raonpnon el a untie futor throughout ibe tat 
lnmi ■ dulhag d. however Thi crttoderetioo of i tat mido up of a tub-tan 
adi wmptod of • different onhopul lutor dfcbori thu the imuaeaoa di 
node letter produced in unuuoue rfdfmte of rilbbifty with ■ nrio of (n-JO/ 
U-l) to du anna tftinuto. Spedil d/Bcohia araka from thw error In ipjdk*- 
tion J current wh/ibua a ihoit n»n « to cot bittonn ire dlecmied Applea- 
dn of due un nudri-foctor concept a ten-inl y t a di ed ora iiraHu difficult*! 
m that Held. Hu Inn trer roefinent ipprouhn VW u in upper limit rather 
then lm^ind ippraichu vtfn n » kmr Unit ji*«r A . Tblt . fagf 
ftadlire Mctmo tor i« wewetunitiiui error in the Kudtr-Rkterd*on lornndi (W 
A uw method of vqIiodi nib tare butd opon tbc frem-uic codtaent » proportd 
ud laitnmfr podmtd. Eitow tob raw method, or i ramplm unor “Hail » 
ncnM u the only proper ippnuch to the profalaa of wr rdabuity, and toe 
{ma-rvJHi/r codScxnt h ibnflirty fotanroemled u the proper ipproech for u«n 
inilyin. (Cnnay AyrWfrtte ) 


IJUfU Choia Gmtf km mn ‘ »«• «'“» 
• — of ihre. iroope. it the U, fl. Niwl Trim me 
ru^fn) b woMhe “filte poelttm? 


on i eanipfi popularem, 

Sawn. Newport, R. L Hu tret m nuania a n»re wn*— • 

■chhu op •M per cent d the poop crf -117 normal wbjjca, It wu uaitnfirtmy 
bpknpi out true pontha, colm* ndy 59 per crau «d ■ armp** "«a «»■ 
drereed m unfit for Nirel rerrie* for neunwthBtiir reuora Fonhawwre, of a 

croup of III nhwn pjwtooily •ifahwl to the otoemtinn — 

*ody hot finely idjodndfic for torrlA,'thr tret pkW up S9 pa 
a the ihnormu innm- The Mtlwn bond the wn “mmlieW* m 
d dndopatpt fw ahw ml«nW RelM /, Sain* 


wild foe anf<4 
want i 

skip 


NEWS NOTES* 

An Immure on « to dent permac) rert wu held on die, Lot Aotelre Cjnnn* 
of the Uniftftfty af GUdarnu florin* the wwt betmnuc July 24, n uiun n 
with the 1944 Summer Bautm 

The minreu wu denened to fidp coUett. ud niumntu. of the wuiem 
inter m the miration ind development of indent penonod nnira. It «» 
pliruted in mDihonrion with Warern Penonod Service, indf ■ cnopennre euoca- 
uoa cf weerern caUeta and onivcmriee formed to worV nfertier on etudent per- 
inajvd pnblrai. The Andtnie Oanndl d Wotere Pemred Semct unda tL 
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ANNOUNCEMENT 


Following an almost unanimous vote by tho mem bon hip 
to the effect that the nominating ballot should be considered 
aa the final yet mg ballot, the following American College Per- 
sonnel Aatoclation officen were designated for the year 
1944-45- 

Preaident: E. G. WiUlnmaon,* Dean of Student*, Unlyftr- 
aicy of MinnBMta« Minneapolis, Minneaou 

Vice-Pret* D D. Feder., Bureau of Naval Peraonnel, 
Washington, D. C 

Secretary! Thelma MiUt 1 t Director of Student Affaire for 
Women, University of Miffaoun, Columbia, 
Minouri 

Treaaureri W W. Blaeaaer,* Admimatrative Secretary of 
the Peraonnel Council, Uiuvemty of Wiacunam, 
Madison 6, Witconam \ 

Members-isLargc of the Eiecutive Council: 

J. L Bergitreaser, Dean of Studenca, College of the City of 
New York. 

A. J Brumbaugh, Dean of Student*, University of Chicago, 
Chicago, Illinois. 

Helen G. Flak, Associate Director, Western Personnel Serv- 
ice, Pasadena, California 

Robert Hoppock, Professor of Education, New York Uni- 
versity, Washington Square, New York, New York. 

Either Lloyd-Jones, Professor of Education, Teachers Col- 
lege, Columbia University, New York, New York. 


* fierriiti too tamsd ym ot » wre-j nr term, 

t The eocrenry b elected for a nro-ynr unn (1944-46) 



A TECHNIQUE for scale analysis 

WARD H GOODEN QU OH, Sct , A.U4 

Ih tha pan two years a considerable amount of rime has 
been devoted to scale analysis by the Research Branch, Infor- 
mation and Education Division (formerly Morale Service^ 
Division) of the War Department, where tho use of scales has 
been important in attitude and opinion research. 

The aim of this pflper is to make available to otherB working 
with Kales one of the techniques for scale analysis developed 
in the Research Branch The theory involved in this type of 
analysis has been outlined elsewhere 1 and is not directly dealt 
with here » 

Four specific techniques have been employed by the Re- 
miich Branch. AIL produce essentially the same results, dif- 
[arias only in the mechahics involved All have limitations 
olhtr of a rigorous or practical nature 

The first of these is the least squares method (Guttman, 
lam, “The Quantification of o Class of Attributes " The Pre- 
Jutlon of Personal Adjustment, Social Science Research Coun- 
cil Bulletin No 48, 1941). This method is too laborious to be 
reidily usable when one ie dealing with more than a few item* 
and categories 

A second method, the one generally used by the Research 
Branch, is to plot data on a Scalogram Board, invented for the 
puiposa by Dr Louis Guttman This is an easily used system, 
especially if the sample is confined to 100 cases or less, which 
Is usually adequate; but it is less rigorous, since manipulations 
of the data are baaed largely on inspection. The board has the 
further difficulty that it is somewhat expensive to construct. 

i Geomn, Look. a A Bilk for Sollof Qua Hath* Dia * American SdekJaf 
faUtmi., K (19*4). 119-150. 

1*? 
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A third method, the tnol-sconng and graphic technique, 
has been used in the work of the Department of Sociology and 
Anthropology at Cornell Univenity as well as in the Research 
Branch. It has the advantage that it does not require a board, 
but is less flexible 

Finally, the tabulation technique, the one to be outlined 
here, was devised to enable scale analysis without a board, at 
the same time preserving its flexibility. It is also more rigor- 
ous than the ScaJogram Board method and can be used with 
any size sample. Its limitation m that, for practical use, the 
relative rank order of categories for each item must be assumed 
in advance, which is not necessary with either the least squares 
or Scalogram Board technique The technique is eaay to learn 
and as a process reveals clearly what is operationally involved 
in Kale analysis. 

According to Gunman's definition,' the multivariate distri- 
bution of a set o[ qualitative items forms a Kale for a popula- 
tion if the following conditions are satisfied 1 

(a) the items have sameness of content (that », they form 
a universe, of content) , 

(h) each item is a simple function of scores derived from 
the distribution, 

Condition (a) is mainly determined by the nature of the 
problem an investigator is interested ul For example, suppose 
we are interested in students’ expressions or adjustment to life 
at a particular college. For this purpose we might ask such 
questions as. 

(1) How satisfied are you with college life! 

(2) Do you think chat what you are learning is worth 
while or not? 

(3) What sort of a time are you having at college! 

We ask these questions because they are a sample from the 
universe of content which is the object of our investigation 
Having defined the universe (in this case, adjustment to col- 
lege life) and having collected a sample of data, we want to 
know whether or not the responses we have obtained behave 

*7ui. 
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aj simple functions of a single variable If so, this will be evi- 
dence chat the universe is a scale for the population, and we 
can denve meaningful relative adjustment Korea bawd on the 
responses to the items 

This brings us to condition (b) of a scale. Can we show 
that each item it Or is not a simple function of scores derived 
from the distribution of the items? The procedures to be dis- 
cussed here refer mainly to this problem, the practical one of 
outlining a specific technique whereby It is possible with rela- 
tive ease and precision to detormino whether or not a set of 
items satisfy the formal condition (b) of a scale. 

The question might arise as to why this u a problem at all 
If a set of items formed a perfect scale, this would be eaBily 
established by any of the several techniques The problem 
arises when the scale-relationship of a set of * terns u obscured 
by the item/ categories This requires the combining of cate - 
gones, which manipulation creates the main technical problem, 

as wifi be seen in the ensuing discussion 

For purposes of illustration we shall concern ourselves with 
tho three hypothetical question^ on student adjustment to col- 
lege life. We shall assume that each question has three cate- 
gories, which we have assigned them on the questionnaire, 

(1) How satisfied are you with college life? 

(a) Very satisfied 

, (b) Neither satisfied nor dissatisfied 

(c) Very dissatisfied 

(2) Do you think that what you are learning is worth while 

or not? 

(a) It la worth while 

(b) Undecided 

(c) It la not worth while 

(3) What sort of a timB are you having at college? 

(a) lam having a good time ' 

(b) lam having a fair time 

(c) I am having a miserable tutie 
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First, then, we must see what the conditions are under which 
each item can be said to be a simple function of scone derived 
from the distribution of the items To do this, suppose the 
responses to our three questions gave us the following marginal 
frequencies! 

Cettgon 

*. k. *, Total 

1 35 20 55 100 

lion 2, 20 60 20 100 

3. 40 30 JO 100 

We can express these marginal frequencies m bat chatt 
form (Fig. 1). 

i 2 a 



For the items to form a scale, that la, to be simple functions 
of scores derived from the hems, all those respondents who 
make the positive response (a) to item 2 must also ouke posi- 
tive responses to items 1 and 3, as Guttman has shown (ibid). 
This group of respondents would constitute a scale type In 
like manner, die other scale types pan be read from tho bar 
chart (provided, the Items form a scale) simply by connecting 
the ban with dash lines (Fig. 2). 
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It ii clear then, that, given the marginal frequencies to a 
■at of items and the relative position of the categories within 
each item, It is possible to ascertain which response combina- 
tions, from among all those feasible for the items, constitute the 
ideal Kale types for those marginal frequencies. 1 But we still 
do not know whether or not the items actually scale We are 
faced with the problem of determining how close to the ideal 


Seels 

l 2 3 Frequency Type 

20 1 

6 2 

16 9 

6 4 

26 5 


10 S 

20 7 


Figures 



Rssponss 

Combination 


b-b-b 


e-e-c 


the obtained response combinations and their frequencies ' 
actually come. 

If a tally of the response combinations obtained showed 
all to coincide with the ideal Kale type combinations and their 
frequencies as determined from the marginal frequencies, the 
responses would form a perfect Kale and there would be no 
further problem of procedure. In practice, however, such 
perfect conformity is almost never encountered. The response 


• It fa (Bailie to diumvln* tfce mufeinoi trasnHa of eali irpw thoiifb nor ilu 
Van Uhu cl mpaiwa whkh corut irate then by Unug Gataan'i farmuk 
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mtfiB munbir oS (rim (“Tb» U 

b* As Kntofle of * Sole," Miwofnphd Pipe, 1 
8am, DiyuJoa. ASF, 1941) 


patterns obtained always vary to a greater or lesser decree I™, 
the ideal Kale established for the marginal frequencies 

The problem, therefore, h to determine to what extent din 
response pattema obtained approwraata the ideal k^ v . 

what extent the response patterns obtained may deviate L 
the ideal scale and still permit of analysis in seals terms an j 
what tbs procedural implications are when the response* □«. 
tenia deviate beyond such an established point of tolerance 
from the initial ideal Kale 

Gunman hss set the point of tolerance for deviation at lSy 
In other words, at least 8S% of the total number of respond 
must fall within the Kale' pattern, id that it fc possible to repro. 
duce at least 85% correctly all the responses of all the respon- 
dents from the scale scores. 

If there is more than 15% deviation from the initial ideal 
scale, ns have evidence that there is more than one factor or 

variable involved This allows us two Inferences, One u that 
the extraneous factors are of minor importance and mainly 
affect only the categories, in which case they can be paniilkd 
out by combining categories, The other inference u that there 
are major factors affecting the items themselves, in which cue 
category combination will not partial them out and no Kale 
can be established^ 

To illustrate what ta involved where the first inference 
obtains, let us suppose that a count of all the possible reaponie 
combinations for our three items on adjustment to college hie 
showed the following results (When large samples are in- 
volved, this can eeeily be done on the counting sorter msdilns 
or with tabulating equipment if the data are put up on L.B.M. 
punch cards. Lacking such equipment, one may conveniently 
determine the nature of die scale relationship by working with 
a sub-sample of 1QQ respondents.) 

Here we have deviation from the ideal scale well beyond 
the 15% pant of tolerance. Our problem now is to examine 
the nature of the deviation. 

A. glance at the uk response combination* which deviate 
from the idaal Kale types shows that four of them have one 
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thing In common i the response (c) to item 2. If we coasider 
categories (c) and (b) on item 2 to be equivalent responses 
from the standpoint of the scale relationship of item 2 to the 
other items, then 15 of the deviant response* coincide with wale 
types; b-c-a is equivalent to the scale type b-b-a, b-o-b to 
b-b-b, and e-c-b to c-b-b Furthermore, the scsle types 
c-c-c and c-b-c ire equivalent and die sum of their obtained 
frequencies u equal to the sum of their ideal frequencies. 

Therefore, in order to reduce deviation from the idea! type* 
and frequencies to lew than 15%, it is possible to combine cate- 
gories of any item. Each combination el minutes a discriminat- 
ing scale type, however, and a mimmtnn of combination h 
therefore desirable.* It ia to be noted that since a combination 
involves the assumption that two categories are equivalent in 
the Kale, It is ordinarily possible to combine only categories 
which are adjacent to each other In the category hierarchy of 
an item. In our present example we can combine positive (a) 
with neutral (b) or neurral (bj with negatlte (c), but not 
positive (a) with negative (c) without including neutral (b), 
which would diminne the item. From the Kale as having no 
discriminatory value. 

To facilitate category combination it ia generally desirable 
to kc up a distribution table rudn as the one below (Fig 3). 
Fat purposes of illustration we shall show m the Cable all the 
possible combinations of response, which ones are the ideal scale 
types, and when the obtained frequencies fall Using our three 
items on adjustment to college life, we get the following- 

•la Kfertaa wiifc Omwub’i fomdi m foeocu 3. 
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When we combine categories (b) and (c) on item 2, as wc 
have already ascertained 'to be desirable, our table looks as 
follows (Fig. 4): 
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In order to reduce the deviant frequency 10 (pr the raped* 
combination c-b&c-a (Fig. 4) we can combine either (b) and 
(c) on item 1 (in which case b-bfcc-a coincides with e-b&e-a) 
or (a) and (b) an item 3. The latter will alio reduce the 
deviation involved in the combination a-*-b. Thu*, by the 
fewest possible combinations of category, we obtain the maxi* 
mum possible reduction m deviation and emerge with the fol- 
lowing table (Fig. 5) i 
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Tie idle that emerge* from the data actually obtained for 
oirthmiteiiu can now bepreiittted in bar chut form (Fig. 6). 
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We can now aay that all thoee reipon dents who are included 
n roponte (a) of Item 2 are alio included in (a) of item 1 and 
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(abb) of item 3, and so on down the Kale. It la possible to 
reproduce the responses of any respondent from whatever act 
of scores one may wish to use to designate the five scalq types. 

In other words, we can now discriminate five degrees of 
what we are calling adjustment to college life op the basts of 
our three items 

1 Those who a) are veiy satisfied with college life 

b) feel that what thqjr are learning u worth 

while 

c) have a good or fair tune 

2. Those who a) are vaiy satisfied with college life 

b) feel that what they are learning u not 

worth while or are undecided 

c) have a good or fair time 

3 Those who a) are neither satisfied nor dissatisfied 

b) feel that what they are learning is not 

worth while or are undecided 

c) have a good or fair tune 

4. Those who a) are very dissatisfied with college life 

b) feel that what they are learning Is not 

worth While or are undecided 

c) have a good or fair tune 

5. Those who a) are very dissatisfied with college life 

b) feel that what they are learning Is not 

worth while or are undecided 

c) have a miserable time ( 

For purposes of illustration, all deviation was reducible m 
our example. In practice, however, complete reducibility of 
deviation Is rarely poaaible without combining all categories of 
all items, thereby resulting in no discrimination at all. 

When the amount of deviation it sufficiently reduced to 
make further combination of categories impractical, there is the 
problem of how to More the few remaining deviants (the devia- 
tion presumably having been reducible to 15% or less). Sup- 
pose we had had such a response combination as a-a-c on our 
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three items by the time we had done all practical category 
combination (Fig. 5). By scoring a-a-c with a-q-a&b we 
would have only one deviant response (c) on item 3. Any 
other scoring, as, for example, with c-bfltc-c, would leave at 
Unit two deviant responses (a) on item 1 and (a) on item 2 
Maximum reproducibility of response from the scale score is, 
therefore, the principle one would ordinarily desire to follow 
in tbe scoring of deviants. (It occasionally happens that a 
deviant can be scored in several different ways and still permit 
of maximum reproducibility. The indeterminacy arises if the 
number of items is small and can be removed by using more 
items. Usually the indeterminacy is harmless, however, and 
it is sufficient to give such deviants the most neutral of the 
several best possible Kale scores.) 

When many items and/or categories, are involved, it la ob- 
viously too cumbersome to work with such tables as those 
above (Figs. 3, 4, 5), where all the fosnbl* combinations of 
response are shown. It is not difficult, however, to work with 
tunilar tables allowing apace only for tbe response combinations 
actually obtained By omitting from the table all those re- 
sponse combinations which have a zero frequency, the resuldng 
abbreviated table is convenient. To illustrate, we could have 
set up the first table in our example with the three items on 
adjustment to college life (Fig. 3) in the manner shown below 
(Kg-7). 
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In the course of our discussion we have seen that it is pos- 
sible to determine from the marginal frequencies of a set of 
items what the scale types should theoretically be for those 
itqms and for the population responding to them. Then, by 
comparing these with the response combinations obtained and 
by combining categories, if necessary, we have observed how 
the Existence of a scale ean be established, how the remaining 
deviation can be determined, and what the principles are which 
govern the s coring of deviants 

In conclusion, it is important to remember an assumption 
which has made possible the procedure outlined Everything 
that we did involved the premise that the hierarchy of category 
within each item was reasonably correct to begm^th, even 
where we made combinations of category If the content of 
our categories had been such that no assumptions could have 
been made as to char relative positive or negative value under 
each item, we could not have set up an initial ideal scale for the 
items, a step on which our entire procedure was dependent 

From the practical standpoint, this limitation is not too 
important for work in the field of attitude and opinion, since 
the order of categories within Items can usually be judged 
beforehand with sufficient accuracy. In some cases, however, 
it may be difficult to do this Any of the techniques for Kale 
analysis can handle this problem of determining category order 
“blindly*, but it will ordinarily Involve far more labor than in 
the case (such as that outlined in this paper) where the cor- 
rectness of a pnnn judgments is assumed. The tabulation 
technique of this paper is being extended to include this mast 
general case where the category order is not known 
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A STUDY O? THE USE Or MECHANICAL APTITUDE 
TESTS IN THE SELECTION OF TRAINEES FOE. 
MECHANICAL OCCUPATIONS 

• j w McDaniel 

BibndtU Junior CnUsp 

AND 

WM.A REYNOLDS 
Saciuusu An Serna Gumnirul 

Wartime dislocation of the labor market m the direction 
of mechanical occupations emphasires a need for jelective 
screening of trainees Postwar return to' trade, service, and 
professional occupations will demand still greeter attention to 
applicants foe retraining- If schools are to retain control over 
their programs, they must provide for increased vocational 
guidance for youth just preparing to enter the labor market 
and Tor adults seeking occupational readjustment. Increased 
guidance demands improved methods for determining potential 
occupational fitness Training programs now in the secondary 
schools give the schools a fine opportunity (or developing elec- 
tion *nd guidance procedures useful during wartime and Indis- 
pensable in vocational relocation after the war. 

The ideal test of job fitness is ^ tryout experience under job 
conditions and under expert observation. The practical limi- 
tations on the wide use of this selective device necesaitatee con- 
struction of substitutes Two general types of ouch substi- 
tutes have been devised One type of measuring instrument 
» a miniature tyork sample in which effort is made to duplicate 
the work situation under measurable condition! Another type 
of instrument attempts to analyze the measurable abstract ele- 
ment* of the job activities, This type may reduce tha situ- 
ation to elements that can be tapped m a language form or 
other visual pattern and results in pencil-and-paper tests of 
aptitude. A number of these tests are on the market, and some 
m 
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of them claim high correspondence with actual work iucu« 
For most of them very limited data ara available showing their 
relationship to tramability of potential workers. The gnat 
need is for specific local evidence of usefulness of such tests in 
selecting trainees for middle level occupations 

This paper reports a study begun in the spring senuat or of 
1942 at the Kem County Union High School and Bakersfield 
Junior College for the purpose of forming a teat battery to 
diet success in mechanical training courses The study *u 
based on the results of administering three tens purporting to 
measure mechanical aptitude to 206 students, of whom 1U 
were in high-school classes and 94 in National Defense Train- 
ing programs. Trainees in machine shop, aircraft engines, air- 
craft mechanics, wood shop, and welding were tested on the 
Bennett Test of Mechanical Comprehension, the MacQuctne 
Mechanical Ability Test, and the O' R our he Test of Mechanical 
A bility, Junior Grade, Form C. • 

The criterion of success used to validate the test* was the 
median rating on four train given the trainee by the instruc- 
tor. After careful instructions tn the use of the mechanical 
aptlmdo rating scale the instructor! rated the trainees on. (1) 
aptitude for learning as evidenced by ability to underatand and 
apply instructions to the joh, (2) speed and accuracy as evi- 
denced by ability to acquire muscular and manipulative i kills, 
(3) workmanship as evidenced by quality and precision of 
work done, and (4) mtereat-entbuBisam as evidenced by eager- 
ness in getting at the job and staying with it. Ten-point scales 
were constructed and behavior descriptions placed underneath 
various points on the scales. 

Results 

Of the 206 students, 197 were males and 19 were females 
The females and those who had incomplete test scores were 
eliminated, leaving 147 males in the validating group The 
means and, standard deviations of the various rests for this 
group are presented in Table 1. The data on the high-school 
students and National Defense trainees are presented sepa- 
rately. Although the National Defense group is superior on 
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the Bennett and O’Rourke tests, the critical ratios between the 
two groups do not exceed 2.7/ and for the purposes of this 
study the two groups may be considered to be comparable. 

Tbo mtercorrtlationa of the subtests of the batteiy ait 
shown in Table 2 Since the standard error of an r of zero for 
147 cases is .082, it u seen that only forty-two of the utter- 
correlations are significantly greater than three times the stand- 
ard error of a chance correlation Howevfer, an attempt was 
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nude to combine tome of the subtest* so that a significantly 
higher nrafuple-correlatiOQ coefficient would be obtained. 

The first combination of aubteau to be tried consisted of 
Eubtescs B,M„ JM, and 0,, aj these had higher coefficients with 
the criterion while at the same tune having lower correlations 
with tha other tern. The corrected 1 multiple correlation coeffi- 
cient was found to be JS, and the combination was rejected as 
being too low in predictive power 

The second battery tried was tbt combination of all sub- 
t 
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tests, excluding tha totals of the MacQuarne* and O’Rourke 
tests. The confected multiple-/? was .43 When the predicted 
criterion scores were correlated with the obtained entanon 
scores tha correlation was found to be slightly, but not signifi- 
cantly, higher, or 45 * Because of the labor in forecasting from 
ten teste and because the efficiency of prediction » but 107% 
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better than chance for this batteiy, a combination was sought 
which would contain fewer teats hut would give at high 
efficiency of prediction. 

The third trial batteiy consisted of the seven test! H, M (l 
M», Mi, Mi, M, and O*. The corrected mulriple-correlarion 
coefficient was found to be .45, and when computed by corre- 
lating predicted with obtained criterion scores, 47. The effi- 
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TABLE 3 







(xncy of prediction for this battery was slightly better, being 
11 7% better than chance, 

A fourth battery was tried, consisting of the five teats B, 
M Mu Mi, and Mi, and the corrected multlple-A was 42, 
1^ correlation between the predicted and criterion scores 
Junred a slight drop to 41 The efficiency of prediction for 
rhu battery wa« thus only 8 79% better than chance. 

Tbs coefficient* of multiple-correlation foi the four tnal 
battens* ai« summarized in Table 3. The efficiency of prtdtc- 
um foreach multiple-^ u shown in tho last nght-hand column 

Since the multiple-correlation coefficient of the third battery 
jgdicMa that it is the beat of the obtained combinations, the 
nttdicted scores were correlated with the obtained criterion 
jam for each separate das* oF mechanical work.* The data 
ffoe summarized in Table 4 

Fran Table 4 it may be noted that the multiple-# varies 
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widely from class to class The coefficient for the high-tchool 
Machine Shop clan, R « .60, is sufficiuntly high for lorecasting 
purposes, while for the National Defense A/C Machine Shop 
class, R ■ - .42, it is useless . An examination, of the means and 
standard deviations of the classes shows that one instructor, 
Ro, rated much more severely than the others, and the negative 
correlation Indicates a different standard of scoring in that 
class. It is possible that the results of the study would have 
been affected in the direction of greater forecasting efficiency 
if the data for this class had been omitted 

Tho small number of cases in the dames, however, should 
bo noted as a factor introducing unreliability in the results 
shown in Table 4, as strict comparability between the instrue- 
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ton* fating* could not be obtained by transmuting them to a 
scale with a common mean and sigma Perhaps more impor- 
tant than this is the fact that the ratings were taken to the 
nearest whole number on a ten-point scale, and were not suffi- 
ciently fine for meaningful transmutation. 

Further evidence for the validity of the forecasting equa- 
tion was obtained by predicting the scores of the nineteen 
women not included in the validating group. Fourteen of them 
were in a dan on Aircraft Engines, under an instructor not 
luted in Table 4, and five women were in the National Defense 
A/C Machine Shop dasa listed in Table 4. The raultiplo-# 
for this group was .78 The coefficient is slightly- greater than 
three times the standard error of an r of zero for 19 cases. 

The regression coefficients (Beta weights) for the tests are 
given in Table 5, > 
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It It evident from the relative magnitudes of the Beta 
weights chat the MacQuamo Tapping and Dotting Teats, and 
tbs O'Rourke General Information Teat contribute the moat to 
the forecasting efficiency of the battery 

Summary and Conchuvms 

1 Three standardized tests of mechanical aptitude were 
administered to 147 high-school students and National Defense 
minces in five mechanical training courses, 

2. From the three major teats a combination of seven sub- 
t«sta was selected which had a multiple-correlation coefficient 
with the criterion of instructors* ratings of .47 It is probable 
that this value would have been raised, If two criterion mea- 
sures had been obtained and an estimate of the true correlation 
of the battery with the criterion had been made. 

3 The forecasting efficiency of the battery differed for the 
five training courses, but its over-all value is significant m show- 
ing that prediction formulae need not be confined to but one 
type of training. Evidence, while not conclusive, was obtained 
tbit the formula would forecast a new group with an efficiency 
greater than if no correlation obtained. 

4 Weightings of the subtests of the batteiy indicate that 
die MacQuame Tapping and Dotting Teste, with the O'Rourke 
General Mechanical Information Test, contribute most to the 
forecasdng efficiency of the battory. 
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A VALIDATING STUDY OF THE OBERLIN VOCA- 
TIONAL INTEREST INQUIRY 
L.D HART 60 N 

Oh 111 la CoU«a 


In the yeah 1927-28 W H. Brentlinger and the present 
author devised a Vocational Inttrajt Inquiry for use in connec- 
tion with the counselling service which was at chat tune one of 
the responsibilities of the Department of Psychology <at OberUn 
College. Inasmuch as a description of this Inquiry may be 
found in Fryer** Maarut-anmu of InUrasti (1), it will be un- 
necessary to reproduce it here. A statement may wall be made, 
however, as to the hunch which motivated the construction of 
this particular form of inquiry. A rather detailed cl aasification 
had just been completed of the occupations in which the Ober- 
Un alumni were engaged (2) This experience suggested the 
desirability of a vocational inquiry from the results of which a 
man’s interests could be classified in terms of the general type 
of function which appealed to him most strongly. A hit of 
twenty functions was therefore formulated, quite a prion, and 
enough items selected to permit a paired companion between 
each function and each of die nineteen others. The Inquiry 
involves, therefore, 190 pairs oT comparisons and yields a score 
in each of the following twenty categories; 
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THE OBBMJH VOCATIONAL INTBUST INQUHY 


The principal u*e made of the Inquiry was as an aid in the 
counselling of student*. With a change w the administrative 
setup, made in 1928, with reference to vocational counselling, 
uie o£ the Inquiry vu discontinued In order that some con- 
clusion might be reached aa to the validity of this form of voca- 
tional inquiry, a follow-up study was initiated in 1941 by E B 
Cnauft and the author Copies of thejblank were mailed to 
178 men who were the original subjects. Responses were re- 
ceived from 121, or 68 per cent of the list It * unfortunate 
that the population ia not large enough to furnish more defini- 
tive information concerning the characteristic patterns of inter- 
est for contrasting vocational groups. The number is large 
enough, however, to furnish some suggestions as to the con- 
sistency of individual interest patterns over the period of four- 
teen years — time enough to permit most of the men to become 
established in their professions 

Validation af the Inqmry 

1. Extent of Agreement between the Patterns of Interest 
Indicated by tks Inquiries of 1917 and 1941, By ranking the 
functions on the basis of the frequency with which they were 
preferred, it u possible to compute rati difference annotations 
between the two seta of choice*. For the total population of 
121 the consistency of choices on the two occasions is indicated 
by an r of .717 =fc .077 (see Table 1) . The correlations for the 
different vocational group* range between ,899, for the 12 men 
m the field of social science, to .282 for five accountants; 
for nine Individuals who could not ho fitted into any group 
classification, from £42 to XJ63 The table also reports the 
correlations for three special aggregates For the nine artists 
and musicians in dua sample, consistency of interest is repre- 
sented by a correlation of £89, for the 27 college teachers, by 
an r of 804, and for ihe 30 men in business and industry, by an 

* f of .668. 

2, Agreement between the 1927 and 1941 Responsei t at 
Indicated by the Most and Least Favored Functions, Table 2 
report* the three most favored and three least interesting func- 
tions, on the occasion of each of the Inquuies, for each sub- 


group This table shows that m the case of those groups for 
which tho correlation between the two response* showed ij 
relation as high as 726, at least one of the functions was amon* 
tho three most favored on both occasion*, and With all but two 
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of the groups this is true of two or three of the functions. 
When one turns to the least preferred functions, it is desirable 
to eliminate from consideration function XIII (Routine Actlvi- 
ties), since that is almost universally unpopular No other 
function i s commonly eschewed, but at least one of the other 
two functions which was least interesting on the first occasion 
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if found at the bottom of the list in 1941 with all groups for 
whom the overall correlations between the two series of choices 
is at least .648, as well as with three other vocational groups 
when the correlation ia lower than chat figure 
* 3 Agreement of the Profiles mth Empirical Judgment or to 
the Functions Characterizing the Different Vo cottons. Etami- 
nation of Table 2 shows that the pattern of preferred functions 
■hows marked contrasts when one vocation is compared with 
another, and that they conform almost without exception to 
common sense expectation Examination of the responses made 
in the 1941 Inquiry show s, e g., that the three functions chosen 
most frequently by the social tcientists are* Diagnosis, Formu- 
lation of Policies, and Research In response to the initial 
Inquiry, this group showed a strong Interest, also, in Evaluation 
of Conduct On the other hand, they displayed least in teres t 
in Routine Activities, Deeds of Prowess, and Profesu'oiU] 
Entertainment. The phyafbians show a distinctly different 
pattern Their initial preferences were for Diagnosis, Re* 
search, and Offering Service or Aid, hue by the time they 
had begun the practice of medicine, the function of Adminutra- 
tionvif Advice had displaced Research The physical scientist* 
■bowed a similar preference for Research and Creative Activi- 
ties, but substituted Diagnosis for their initial choice of Practi- 
cal Construction Problems. The group of college teachers who 
were working in the general area of the language* report una- 
nimity in their first three choice* on both occasion*; Creative 
Activities, Research, and Evaluation of Achievement or Prod- 
uct. The types of activity most attractive to the clergymen 
are' Oral Presentation of Reports, Evaluation of Conduct, and 
^ either the Formulation of Policies or Arbitration (jhe settle- 
ment of disagreements). The final preference* of the lawyer* 
differ widely from the undergraduate choices. They present a 
logical combination. Arbitration, the Administration of Advice, 
and Teaching. 

On the other hand, there are instance* in which the pattern 
chosen seems unrelated to the occupation. This is notably tb* 
case with the men in the fiald'of secondary education, whose 
more recent interests lie in the field of Creative Activities, 
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Research, and Diagnosis (of difficulties) The teaching func- 
tion does not appear among their most favored activities 
Review of th«r vocational careers ahowa, however, that, in half 
J the cases, die teaching profession was chosen only after 
another vocation had first been tried. Again, one would expect 
Financial Speculation (activities aimed at financial profit) to 
appear among tho most interesting functions of the business 
This appears to be true, however, only in the initial 
faulty, and then only with the advertising managers. These 
facts suggest die desirability of an analysis of the Itema which 
ffcre chosen to represent those functions to see to what extent 
they are twjr representative. 

An Item Analyst! 

In order to lay the groundwork for a possible revision of 
thtMgwty £ he individual items have been examined, first, for 
the purpose of eliminating those that are non-differentiating, 
and, second, for the purpose of selecting those representative of 
the functions having the greatest predictive value. The non- 
differentiating items were located by the process of computing 
fle proportional preference given to each item in the 1941 
jtpuy. Of the 380 items in the Inquiry, 184 were preferred 
hj 70 per cent or more of the subjects and were therefore elimi- 
mted Then, after the subjects had been grouped according 
to the vocation in which they were engaged m 1941, the relative 
popularity of the 196 items was computed for each vocational 
group. Examination of these proportions revealed 45 more 
items which were not distinctive of any vocation in which these 
particular men were engaged Their elimination reduced the 
total to 151 items One consequence af this procedure was the 
reduction of the number of items for some of the vocational 
categories to less than 16. The number of categories having 
ac least 16 items was six, the largest number of items for any 
categoiy being 37 The identifying names being used for theso 
catepiritt are 1) Commerce; 2) Merchandising, 3) Law and 
Social Science; 4) Medical and Social Work, 5) Physical Sci- 
ence, and 6) Literature, Because of the inequality in number 
of items under each category it became necessaiy to use pro- 
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portions rather than sumifiationi of items in making compari- 
sons of relative interest for the different activities, 

The next step consisted in ra-sconng the original responses 
on this selection of items to determine the extent to which the 
respondents' original answen would have predicted their final 
vocational choice At this stage the men engaged In Art and 
Music were eliminated because too few items won found which 
differentiated these vocations. This elimination reduced the 
test population to 112. (It should be noted that in order to 
re-scoro on these 151 selected items it was necessary to cany 
104 pairs of items— including 57 undifferential items.) When 
the original inventories were re-seared on theso 151 items it was 
found that 63, or 56,2 per cent of the men had the highest score 
in that category in winch it would be most logical to dainfy 
the work in which they were actually engaged in 194L 

Further examination of the items suggested the possible 
advantage of making a still more drastic selection of items. In 
order to equalize numbers it was decided to select just tbe ten 
items in each vocational category which were most highly dif- 
ferential For this pare of the study use was also made of the 
responses made on the original Inquiry by two additional 
groups of subjects, 27 studenti in the Hamma Divinity School, 
and 90 secretaries In city YMCA's m Ohio. The responses of 
these groups were studied with the possibility of finding items 
specific to the field of religious work. This quest proved, how- 
ever, to be fruitless 

With this final item selection, consisting of ten items for 
each of six categories, the Oberlin graduates who had responded 
to the second Inquiry were again tabulated on the basis of their 
first choice among the six vocational categories. (In addition 
to there 60 significant items it was necessary to retain 32 indif- 
ferent items, thus making a total of 92 items, or 46 pain,) 
Classification of these men under these six categories on the 
basis of their actual final vocation shows that 56.1 per cent of 
them engaged in fields in the area in which they made the high- 
est scores as seniors. 

The vocational history of 47 other men who had filled out 
the Inquiry is seniore, but wbo had not responded to our second 
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Inquiry, being available, their original responses were scored 
with the 60-item form The results m the case of these men 
give a 66. 7 predictive percentage, 32 of the 48 men having 
chosen on occupation in die one of die six areas for which they 
•cored highest in the Inquiry, 

As a final step m the validating analysis, the 60 items of the 
abbreviated Inquiry were classified on the basis of the twenty 
functions which had been used in constructing the original /n- 
ftury When these were parcelled out in terms of the patterns 
characterizing the interests expressed by the various vocational 
groups (see Table 2), the number of items for each vocation 
proved to be too small to yield significant differential scores. 

Summary and Conclusions 

1. A validating follow-up has been made of the Ob trim 
Pocsu and /fttewt Inquiry, after an interval of fourteen years, 
based upon the responses of 68 per cent of the men tested in 
the year 1927-26. In an item analysis use was also made of 
retpoaies from 90 secretaries of urban YMCA's and 27 students 
in a theological seminary, 

2. When the correlation is computed between the rank order 
of interest expressed on the two occasions, 1927 and 1941, m the 
vocational functions sampled by the Inquiry, the coefficient for 
the entire group of 221 u 717 

3. Examination of the patterns of preferred functions shows 
marked contrasts between the different vocational groups 
represented, and agreement with common sense expectations 

4. Preliminary steps were taken toward a passible revision 
of the Inquiry ui the form of item analyses aimed at eliminating 
noo-diffwentratmg items Thu process showed that, on the 
bans of the interest shown m the particular items used ih this 
Inquiry, there men fell into six categories which have been 
labelled as follows 1 1) Commerce, 2) Merchandising, 3) Law 
•nd Social Science, 4) Medical and Social Work, 5) Physical 
Science, end 6) Literature 

5. The item study revealed 151 of the original 380 items 
which had been differentially selected by the members of there 
m vocational groups When the original 1927 Inquiry was 


re-sepred, using only these 151 items, S62 per cent of the men 
made highest proportional scores m the vocational area in which 
they subsequently chose their life work. 

6. The ten most highly differentiating itema were selected 
for each of the six vocational categories, this constituting a 
further reduction of the total number of items to 60. Scores 
obtained from the origins] responses of the 121 men on this 
60-item Inquiry give first choice to that category which most 
closely corresponds to the final vocational choice in the case of 
56.1 per cent of them. 

7 Finally, the responses made m 1927 by 47 men who did 
not answer the second request were scored on the 6Q-itcm In- 
quiry, and with this group it was found chat, m two-thirds of 
the cases, the first choice was given that categoiy represented 
by the vocation subsequently chosen, 

8. It is to b« regretted that the small size and apecialtzcd 
character of the population involved In this study precludes the 
me of more refined statistical procedures For Item differenti- 
ation (3) The results, however, appear to harmonize with 
those obtained by assigning multiple regression weights to the 
items. They furnish further evidence, moreover, pf the possi- 
bility of predicting vocational choices of college seniore within 
areas which stress particular functional interests. 
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GUIDANCE SURVEY OF STUDENT PROBLEMS 

ELIZABETH GORDON ANDREWS 
Florida Stale Colltga (or Wo men 

This investigation was undertaken as a contnbution to a 
curriculum irmly being made by a sub-committee of tho 
Faculty Committee concomed with the functioning of person- 
nel work and guidance on the campus of the Florida State Col- 
lege for Women. The students who secured the data were 
graduate students in a class in Personnel Work in l941-'42 
These investigators visited as many of the girls u were 
found in their dormitory rooms when called upon All students 
at the college reside in dormitories (or in approved itudent 
houses). The dnnnitories were divided between the two in- 
vestigators. If a student was not in her room at the time the 
visitor called she was not included in the study. A total or 
279 students, or 14 per cent of the total student body, were thus 
interviewed, i 

Each interviewee was presented with 1 a sheet entitled “Gui- 
dance Survey." On this sheet were indicated the purpose of the 
study, a suggestive list of problems which the student might 
have had since entrance to college, a list of the possible soured 
of help that she might have consulted, and an example of the 
information desired. This study does not include the numerous 
times students are called to administrative and other offices. 
We limited our study to problems on which students volun- 
tarily sought assistance, 

Only those problems wore recorded which assumed some- 
what serious proportions For example, the investigator would 
■ay, “Hava you had any health problems since you have been 
in college? By health problems, we do not mean the times you 
may havo gone to the InRrmaiy to have your throat swabbed, 
but instances in which your health was such as to definitely 
sos 


210 EDUCATIOHAL AKO PSYCHOLOGlCAt MBAHUBMHTf 

**?“ * mr * d ™? W ° A ;“ DuciplHWpy pmMot, 
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problem, are defined on tbe sample .heel ™ type * •* 
One of tbe chief faotnn m causing unreliability ol „„„„ 
duo to the uncertainty of memory For nuunpt fn 2”* 
report homeeickneeB ae a problem mom than upper-J^Z? 
However, the. may be partly due to the .election diet ’ 
and the elimination of the homeeiclt ae „,l| ,, ™ ,IB 
students. Poorer 

Sources of Half 

Table 1 mdieate. the frequency with which the various 
office, or individuals on the campus served the students mu. 

TABLE 1 
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Viewed with regard to their problems 

Both upperolaramen and freehnien report that they 
the Regietrar’. office eolely regarding auch problem. „u„e- 
credit., schedule., cuts and euch marten i. pertain to leZ 
tration 

The Business Office is sought by both groups chiefly y^tb 
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regard to financial matters related to college accounts, Twice 
they reported seeking aid in personal problems but in both 
these cases the student had a personal friend in this office. 

The Dean of Student's Office functioned chiefly in three 
areas insofar as students voluntarily seek help from this office, 
extra-curricular problems, personal problems, and discipline. 

The four areas in which the Personnel Office was most fre- 
quently sought were educational, financial, vocational, and 
placement problems While personal problems were seldom 
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Urted rh« fs perhaps due to the objective approach to problems 
m tbit office which avoids labelling personal problems as such. 
Placement is pemdrariy a problem of upper -classmen, especially 
oT seniors; hence it ja w be expected that freshmen would not 
be concerned with this function of the Personnel Office. With 
this exception, upper-classmen and freshmen consulted the 
office most frequently on the same problem areas. 

J rire Infirmary was, of course, sought mostly in matters or 
7 rWl reloted problems In which advice was 

sought from the physician or nunc, 

TVDffice of die Director of Residence was sought a limited 


number of times chiefly in regard to homesickness, roommate 
difficulty and occasional personal problems 

According to student reports. Social Directors am .ought 
infrequently, bu,t on a widely divergent list of problems includ- 
ing educational, religious, financial, hoalth, vocational, personal, 
and disciplinary problems. It seems that the type of service 
the Directors in dormitories render most to the students lies m 
the field of personal problems. 

Religious Secretaries are sought by students chiefly with 
regard to religious problems. The number of times the Secre- 
tary is sought seems to be much greater with upperclassmen 
t han w ith freshmen This may indicate that the Secretaries 
become increasingly helpful as they coma to know the students 
longer, or it may indicate that religious problems are more 
prevalent among upper-classmen than among freshmen. A 
previous study on this campus would, however, indicate that 
the problems in this area are more prevalent m the earlier yean. 

The Student Couneelloix were seldom sought regarding 
financial, religious, health, and placement problems Their 
chief service includes assistance m educational, extra-curricular, 
and disciplinary problems 

Educational, personal, and vocational difficulties constitute 
the three outstanding problems about which Faculty Advisers 
are consulted. The small number of rimes Faculty Advisen 
were sought voluntarily was somewhat disturbing, but no 
doubt one cause of this is the difficulty of finding an hour at 
which the student and the Faculty Adviser have corresponding 
free time for conference. Thu again points to the need for 
Faculty Advisen to be given a reduced reaching load in order 
that they may devote more tune to counselling. 

It is evident from the itudent reports that Academic Deam 
assist students mainly with vocational and educational prob- 
lems. This does not indicate vocational guidance so much m 
counsel with regard to pre-professional courses, such at fft- r 
medical courses, courses for nurses, teachers, etc, 1 

Freshmen consulted the Heads of Departments only qn ,i 
educational problems while upper-classmen conferred 
them on problems of finance, extra-curricular acdvfdtf, ipig#} ^ 
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mane, and vocational guidance. This is probably due to the 
fact that freshmen have very little occasion to become ac- 
quainted with Heads of Departments until they have chosen 
their major field, which frequently is not until the latter part 
of the sophomore year. 

Teachers other than Faculty Advisers make a valuable con- 
tribution to the guidance program by giving satisfactory help 
on educational and vocational problems. They are also often 
■ought by upper-classmen regarding placement problems. 
They may be sought m any or all fields of guidance, but are not 
called on frequently in problem areas other than education and 
the vocations. 

If the nine types of problems were ranked according to the 
frequency with which students reported that they sought help 
(Hi these problems, the results would be as givdn m Table 3. 

TABLE 3 
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sophistication of the upper-classmen. Vocational problems 
rank high with both groups, indicating that selection of voca- 
tion and preparation for a vocation are not confined to the first 
year but are also problems during the later years of college. 

Problems of discipline, religious and extra-curncular matters 
are not so great a source of disturbance apparently as are other 
problems. Perhaps students fail to recognize them as prob- 
lems and hence do not seek help with them. Students seek help 
most in educational, vocational, personal, financial, and health 
problems The attention of Counsellors might be directed to- 
ward improvement of assistance in those areas m which sui- 
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From Table 3 it is apparent that the problems that trouble 
both groups most are the educational ones, and this is the field 
in which students fool the most pressing need for assistance 
F rah men appear to need more help than upper-classmen with 
personal problems Perhaps this is due to the more obvious 


dents report seeking help most frequently, and further investi- 
gation should be made to determine whether problems actually 
exist in other areas where students are unaware that they need 
counsel 1 

During the interviews, forty-eight students reported that 
they had had problems about which they had not sought help. 
Twenty of these cases were personal problems, the remainder 
being distributed over all tho other fields It » apparent that 
personal problems constitute the oustanding ares in which stu- 
dents realise that they need guidance but hesitate to ask for 
help 

Out of the sixteen possible sources of official assistance avail- 
able to students on the campus, this survey reveals that the six 
offices or individuals sought most frequently by students in- 
clude m order of decreasing frequency (1) Office of the Director 
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of Penonnel, (2) Teachers other than Faculty Advisers, (3) 
Academic Dfeana, (4) Faculty Advisets, (5) Social Directors, 
and (6) the Infirmary. 

In Table 4 it will be noted that the actual problems in which 
help was sought by students during the period of time covered 
by the survey are very much in the same order as reported by 
the students, indicating that the reports were fairly accurate 
and that the sampling by the method used was quite repre- 
sentative, The only change in order would be placing the Per- 
sonal problems below Vocational and Placement problems. As 
stated previously, this is probably due to the approach in the 
Personnel Office which seeks to minimize the “personal” prob- 
lem in favor of a more objective approach At the same time 
the confidential records of the office indicate the persona] prob- 
lem involved. 


INTERPRETATION OF THE KUDER PREFERENCE 
RECORD FOR COLLEGE STUDENTS OF 
HOME ECONOMICS 

BOTH T LEHMAN 
The Ohio Euu Uwrcnity 

For the past several yean, die Kuder Preference Record 
has been used experimentally with freshman students, in con- 
nection with a rather extensive guidance program m the School 
of Home Economics of the Ohio State University. This experi- 
mentation has shown that the Record mikes a helpful contri- 
bution to the total picture of the individual which is needed by 
the counsellor, suggesting both vocational and avocationa] 
interests. 

However, interpretation of interest profiles has presented 
serious difficulties. Some hive assumed that, since the home 
economics curriculum bat traditionally been built upon a base 
of natural science, performance interests in that area should be 
strong; that, in (act, students who have little interest in scien- 
tific activities should he barred from entering or continuing in 
the field of home economics. Yet interest scores in science and 
marks received in science courses by home economics students 
have been found frequently to be in direct contradiction Simi- 
larly, one might expect to find a pattern of interests which is 
characteristic of home economists. Such has not proved to be 
the case. The weakness of the assumption is evident when one 
surveys the diverse fields which the graduate jn home eco- 
nomics may enter She may become a teacher, an extension 
worker, a hospital dietitian, a foods manager in a restaurant, 
or she may enter one of the numerous divisions of business or 
of social service. Obviously, it would be dangerous to base 
interpretation of an interest profile either on one's prejudices, 
or on a. guess as to areas which are significant for specific occu- 
pations requiring a home economics background. 

nr 
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Such difficulties In interpreting the Record blocked its effec- 
tive use with college freshmen in home economics. Therefore, 
the teat wu given experimentally to a sample of homo eco- 
nomics teachers, dietitians, and home economists who were 
employed by utility and household equipment companies 
Findings from this preliminary work indicated that there were 
real differences in interests of the three groups and encouraged 
an extension of the study. Other cases were added during the 
school year of 1943-44 

The Study Reported 

To date the Record has been checked by 239 home econo- 
mists. These are distributed as follows. 125 secondary-school 
teachers, 42 hospital dietitians and 72 home economics women. 
In husmess Of the teachers, 51 were volunteers from a summer 
institute held for Ohio vocational teachers; 74 others in the 
State were reached through meetings called by their city super- 
visors The hospital dietmons were secured through group 
meetings in several Ohio cities, and through a random sampling 
of the Ohio Dietetic Association. The business women were 
from larious parts of the United States, They represent a 
random sampling of the extensive membership list of the Home 
Economics in Business department of the American Home Eco- 
nomics Association, supplemented by a smaller number of Ohio 
women not included in this list. Of the business women, 10 
were engaged in restaurant or tea-room management, 11 m 
journalism, and 20 in foods promotion or experimental work, 
and 31 were employed by utility and household equipment com- 
panies All three groups of home economists represented a 
wide range— but ■ lairiy good balance— in terms of experience 
in their respective fields (Table X). 

Farm B of the Ruder Preference Record was used, except in 
the early part of the study when only Form A was available 
In order that all data might bo used, the raw scores secured on 
Form A were changed to comparable scores an the later form 
by means of a transmutation table provided by the author of 
the test. Medians were found in each interest area and their 
percentile equivalents located on o profile sheet. Mention 
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ihould be made alio or tho fact that only a part of tha total 
group received acorn in the mechanical and clerical „ u ZZ 
these areas were not included in the enriy form (Sec footmn. 
to the vanous tables ) 

While the Intcreet patterns of different persons aa revcalod 
by the teat were highly individual, there appeared to bo quin 
d.itinct differences among the three group, of homo econombt, 
mdudad in the study (Table 2). The teach™ and dtetttian. 
had high social service interest*; the business women were 
somewhat lower. Tha dietitians wore outstanding m science 
interests; the other two groups were lower, although still above 
the average for college freshmen The dietitians were also 
TABLK 3 
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much higher in the computational area than were the teacheca 
and business women. The teachers surpassed the others in 
artistic interests, the business women, in the persuasive or gell- 
ing area 

In other words, the areas of highest interest for the teachers 
were artistic and social service, median percentile scores in those 
areas being 71 and 83, respectively In fact, over three-fourths 
of the teachers (77 and 78 percent, respectively) had scores at 
or ahove the median for college freshmen in these areas. Thbra 
wa» some evidence also of scientific interest, but this was not 
nearly so high ss in the cue of the dietitians A breakdown of 
the teacher group In terms of years of experience ahowa that 
this pattern of activity preferences is consistent (Table 3). 

TABLES 

J/rdiM PerteetiU Stent of Hem Amumla Ttedntt wkrA CUiAfei at to 
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An apparent gain in artistic interests and loss in the social ser- 
vice area in die case or the teachers having the longest term of 
service may be revealing. 

The hospital dietitians, though fewer in number than the 
teachers, also presented a clear pattern of intereits Out it end- 
ing areas were computational, scientific, and social service 
Median percentile scores W these areas were 63, 82, end 80, 
respectively. Moreover, around three-fourths of the dietitians 
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had scores above tha 49th percentile (69, 76, and 79 per cent 
in the respective areas). 

Interestingly enough, though the business women differed 
in certain respects from the teachers and dietitians, they showed 
no definite preference pattern Interests in as many as five or 
six areas appeared to be simdar (Table 2), However, when 
this group waa broken down in terms of occupation within the 
business field, differences did appear (Table 4) Certainty u 
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to how real these differences are will depend upon evidence pro- 
vided by additional cases, for the numbers representing the dif- , 
ferent occupations were small. Interests apparent m restaurant 
and tea-room managers were computational, scientific, and t 
artistic (71, 64, and 76 being the respective medwnp, and 70j { 
90, and 60 being the respective per cents above the 49th per- 
centile norm). Similarly, apparent interests of the women in J 
foods promotion and experimentation were m the scientific area < 
(median percentile 78, with '80 per cent above tha 49th pw» 1 
centile norm), of women in home service and equipment, in 
lutsive ( oelling), social service, and probably mechanical 1 

(medians of 68, 68, and 62, with 68, 68, and 79 per cenr °f 
cases, respectively, above 49) . Interests of the journalists 
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centered in literary ond artistic areas (medians of 71 and 78, 
to each case 82 per cent being above 49) * 

Use of the Findings in the Guidance of College Horn* 
Economics Students 


The foregoing findings suggest certain conclusions concern- 
ing the interpretation of interest profiles for homo economics 
students 

1 The study does not reveal a pattern of Interests which a 
characteristic of home economists as a group 

2 However, the student who is considering home economics 
as a field or who u already enrolled In it can profitably compare 
her own interests with the outstanding preferences of those 
actively engaged In various professions requiring a home eco- 
nomic! background 

5 Such comparison should serve as the basis for a discus- 
sion of specific activities In the field being considered which call 
for certain interest patterns Far example, the scientific inter, 
cts of the hospital dietitian are clearly used m the planning of 
special diets based on exact scientific facts, her computational 
interests in the keeping of necessary accounts, the estimation 
qf diets of given composition, and the ordering of food supplies. 
Similarly, the artistic and social service interests of the home 
economics teacher should be related to creative activities in 
home making and to a desire to work with others and help 
them. 

4, Ideally such discussion should eventually lead to explora- 
tory experiences on the part of the student to discover her abili- 
ties and whether she really enjoys such activities Perhaps this 
should follow particularly in those cases where test results and 


professional ambitions are in evident conflict. 

5. In any case, however, the interest profile should he used 
m relation to other factors, — the student's personality, intelli- 
gence, special aptitudes, handicaps, health, and educational and 
expenential background. Left to herself, the student u likely 
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to rely too much on test scores Often she would like a teat 
which would make decision on her part unnecessary. On the 
other hand, the inexperienced counselor with too great a respect 
for tests may use the profile in a dogmatic fashion or without 
supporting evidence. 

6. Furthermore, it is important for both student and ad- 
viser to interpret the Preference Record in terras of what it 
claims to measure, namely, activity interests, not ability, nor 
appreciation. This has been found to be doubly important to 
remember in the artistic, writing, and musical performance 
areas, where ability or appreciation ere so readily assumed to be 

the interpretation. 

7> The counselor must keep in mind also that, although 
certain interest patterns appear in studies of given occupational 
groups, profiles Hre after all highly individual, reflecting both 
possible vocational and a vocational interests, and may change 
with experience and maturity 

Further Work Needed 

This account of the present investigation is in the nature of 
a progress report. Need for further study is evident In the 
fim place, the validity of the tentative findings for dietitians 
and home economists in business must be tested by increasing 
the number of cases Then, too, data from other related pro- 
fessions— such as home economics extension, and the textiles 
end clothing area of business— should be added, if the findings 
are to be of greatest use to the counselor of home economics 
students 

Finally, certain general questions need to be studied, 
Among them are two which face every counselor who attempts 
to interpret students' interest scores One of these is the big 
question of how much student activity preferences change dur- 
ing the four years in college and what are important factors in 
such change The other is the basic question as to whether the 
pattern of interests revealed in a study of professional workers 
is closely related to experience m and knowledge of the job, or 
is more a reflection of the selective effects of the specific under- 
graduate curriculum which waa followed. 


THE USE OF STANDARDIZED READING TESTS IN 
TEACHERS COLLEGES* 


ALFRED M. EWING 
Ton WoCcyan CaUtf* 


What is the most efficient way to use the rcsulta of reading 
tests? What ia the most usable tesc in the reading field? What 
disposition is made of reading test scores? In an attempt to 
answer these and similar questions concerning the use made 
of standardized reading teats in teachers colleges, dare were 
gathered by means of a mimeographed two-page twenty-seven - 
item questionnaire which was sent to 167 teachers colleges 
One hundred three returned the questionnaire, making the per- 
centage of returns, 61.7%, more than twice that of similar sur- 
veyi in colleges and universities No reading tests were given 
-in 17 of these institutions, or a program was just beginning. 
Thus the data here given are from 8 6 institutions answering 
questions concerning the use made of reading tests and the use 
and interpretation of the results. 

The 86 teachers colleges using reading tests reported the use 
of 27 tests. Four tests, the Iowa Silent Reading Test, the 
Cooperative English Test ( Form C2, Reading Comprehen- 
sion ), the N els on-D army Reading Test, and the Entrance and 
Classification Test forT tochers Colleges Entrants, English T art, 
are used much more extensively than are any of the other 23 
testa. In fact the four tests make up nearly three-fourths of 
all the tests, their respective percentages being, 35.5%, 20.0%, 
10.3%, and 6 8% Thu limited use of a wide variety of tests 
tl accounted for by local use of the test, use by its author, or 
some other specific reason and not by general considerations 
such as ease of scoring, popularity of the test, or number of 
■kills or abilities tested. 


* Thk irtjde It bu«d on ■ nudy dona id partial fulfiUmeot rf tha mulwunt 
•w CbimtE si Donor al Edufldoc at Colorado Into Csttlto d SdnnooB, Ora- 
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Reasons for Selecting Tests 

Hie ability to diagnose reading difficulties is given as the 
main reason by 16% of the teachers colleges far selecting the 
reading tasta which they administer to their students. Follow- 
ing closely is the fact that it “meets their needs," 14%, Sim- 
plicity of scoring, 10%; high reliability and validity, 9%, and 
recommendation or selection by authority, 9%, are other 
reasons frequently given by teachers colleges for selecting spe- 
cific reading teats. Twenty-two other reasons for selecting 
reading teats are mentioned once or twice. 

practices m the Techniques of Testing 
Four departments, psychology, testing, English, and per- 
sonnel and guidance, administer the reading test in half of the 
teach era colleges. In the other half reading tests are adminis- 
tered by ton other agencies. 

The faculty ranks first as the group most often scoring read- 
ing teats in teach era colleges. This includes such depirtments 
as English, psychology, etc. There seems to be, from the data 
obtained, no one department by which the scoring is generally 
done outside of die faculty and assistants. 

The personnel and guidance office ranks first as a place 
where reading test scores are filed The deans’ and registrars* 
offices rink next while die English department ranks fourth. 

The personnel and guidance office alio ranks first as a place 
where the scores are available or given out* The adviser and 
dean rank second and third, respectively.’ In right schools 
scores are not given out, while in many other schools more than 
one department gives out the test scores 

Fifty of the 86 schools give test results to din students. In 
three of the 50 schools the test scons are given only occasionally 
to the student and in sk other schools they are available to him 

The Use Made of Reading Test Scores 
The purpose of obtaining reading test scores, os stated by 
40% of the teacher* colleges, is to find the students who need 
help in reading. Twenty-two per cent more say for diagnosis, 
while 16% give appraisal of the student’s reading ability as the 
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scorea. Very few bona fide low scores exist and they an uiua% 
caused by laclc of speed. 

In College D teat scores arc used as one of the bases for 
advising students relative tp the academic load they should 
cany. Instructors may consult records to determine probable 
student abilities. Deans and Advuere find the records vain- 
abla in giving them an index of the students' probable success, 
and a limited number of superior students are granted a partis] 
tuition exemption. They prefer to select superior students 
rather than attempt to save (for teaching) those of inferior 
ability and attainments 

As an example of the use of reading tests m solving per- 
sonnel problems, one of the teachers colleges makes the fol- 
lowing statement from a five-year survey in the college. The 
college records data from several .teats including one on the 
measurement of ability in reading. From tbs great vatu- 
tion in reading ability discovered, certain questions arose. 
What curricula do these poor readers enter? How long do they 
remain in college? What kinds of grades do they make? 

The findings were; Curricula — physical education, agricul- 
ture, home economics. No one chose art, biology, chcmutiy, 
English, music, or physics as his major field. Yeart In col- 
lege — the mortality rate of poor readers is exceedingly great. 
Only 70% are in school at the end of the first year, 45% at the 
end of the second year, 20% at the end of the third year, and 
15% it the end of the fourth year. Grades — those chat remain 
make grades comparable to grades of other college students. 

Plant Used fa th a Remedial Reading Program 
Twenty of the 86 teachers colleges have specialized reading 
classes for improving the reading ability of the students; fifteen ' 
have both a specialized reading clan and a reading clime, while 
eleven report only reading clinics Thirteen schools use special 
sections of certain courses, while nearly equally as many appir- 
J autly give no concern to reading. The clinic jind special cluies, 
however, account for over 51% of the 86 schools whose atten- 
tion a drawn to actus! improvement. 

Small schools, that u those below 500 enrollment, give 
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.lightly more attention to reading than Jo larger tchoob, hut 

the difference 11 not ilgnibcaot _ 

How some of the plans for improving reading actually oper 
etc it ran horn the example, given, one oath for • reading 
dink. . laboratory, an mrliviiiiial contarence method, Kid m 

orientation courae. 

Prom a r«t*«g eKofo-One o! the lean gratifying aepect. 
of tho program ha. been the rnierntninty of renin. In aome 
initaneea waiting claaaca all day, conducting a conlenmra with 
teacher, after achool. and ipcndrng eatra bout. In eonvenation 
have produced no tangible evidence of change in the instruc- 
tional program. However, moat teacher* ihow a keen interest 
in the questions discussed. 

But tangible results are sometimes, oftentimes, almost over- 
whdmiAg. Witness the case of Miss F, a college girl who made 
the following record from tho analysis of her eye movements as 
photographed by the ophthalmograph, 

■Mi Ju*7 Ftb.26 Met H 

te£T**'T\ - | | g 

aJKSTwiI W»S ’ ■ V ’ *4 

Arwift fcnwm d ixjtkw b KcOfldJ 


The increase of 94 words per minute can be readily appreci- 
ated when practically applied to the work of a student. That 
mean* 5640 more words per hour provided die rate can be main- 
tained, and assuming aha reads live bwi* u day, five dayi a ^ 
week, her inereued reading schieveraent rises to an, impressive 
141 p00 words per week, or oyer four books of 30,000 words 

A laboratory reforu tbit the won! ‘laboratory" was used 
became it implied a broader, mare general use than the word 
“dalle" and did not limit the function to disguosu and correc- 
tion. Theroembereof *eda*sleam how to ««up wd main- 
tain cue histories] how to operate certain simple equipment, 
hmv to administer, score, and interpret many kinds of tests; 
andhow tn locate and assemble this essential Information ?nd 
data in practical situations- Students leim to think of tests as 
olds and guide** not ai Infallible conclusions. 
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In a cue of individual help the director of reading improve- 
ment listed the lowest quarter of the freeman class for ipetia] 
help M indicated by results of the loan Reading Tut, then 
interviewed the staff for freshman reading assignments in col- 
lege subjects, raid the assignment himself, and made objective 
comprehension questions on the content. The students read 
the assignments silently in the presence of the director, they 
were told to read carefully but ware urged not to huny. Then 
they answered from what |hoy remembered At the end of a 
time another form of the reading test was given. The improve- 
ment was statistically significant. This type of work is good 
but it is not always possible to schedule a special class. Lntls 
extra time waa required Tor they read the assignments that they 
had to read anyway 

One teachen college used the orientation count to improve 
reading ability under the following topical ideas: Dixullion of 
raiding as a tool, relationship of speed and comprehension j 
administration of timed informal teats— the students were re- 
quired to find specific answers to questions in their assignment; 
use of association tests, in which the student is given a word 
or two snd writes all he knows about these words during a 
minute or two, development by students of informal tests, true 
and fake, completion, etc , use of assignments to provide for 
individual differences by noting books that are easy or difficult; 
check of Visual and auditory efficiency of students whose rare 
is low. 

Recommendations 

On the bane of tho data obtained from the coojjmimi 
teachen colleges, the following recommendations are m»«‘ 

1. More teachers colleges should stress rwnedial reading. 

Forty-one of the 86 schools said they felt their reading program 
was fairly •satisfactory, 33 were dissatisfied, eight wen i unde- 
cided, three did not indicate, and one school said it bed no 
problem in reading. * . . 

2. There is a definite need for both a good achievement teat 
in reading and a good diagnostic test, 

3. Remedial reading should have a ylasaroom sppw« E y 
well si a clinical approach for both speed and compiwsoswV- 
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, Jjf^rked improvement can be made without the uae of 
apparatus such as metro noacopea, tachistoscopea, 
(jT^vd corrective reading manuals, by giving a little time 
jTpnetJce, drawing attention to a few efficiency devices In 
p-mr wading, and ualng mimeographed materials 
5, Follow-up tests should be administered to determine the 
ptmunency of the results of remedial reading programs 
fiTffitb jgu attention than one realizes reading can be 
apioved For example, the student’s attention can be drawn 
to ducarding useless mouonb, such as pointing, head or lip 
morenKnti, at whispering, to making fewer and shorter fixa- 
tjont to seeing phrases and sentences at a glance, to antict- 
pitmg what the author is going to say instead of passively 
([tempting to absorb his idoas, and to practicing reading at a 
4 Waiter than convenient" rate 


MEASUREMENT IN THE CONTINUOUS SELECTION 
AND COUNSELING OF STUDENTS IN A COL- 
LEGE OF PHYSICAL EDUCATION 
AND SOCIAL WORK 


J E. TODD* 

'nulutiRin of Piper Oummtry, Appleton, Wbcenna 
Springfield College u seeking to solve through research 
and measurement some of the problems confronting it in the 
selection and training of leaden of youth for positions as 
teachers, program director*, and directors of social agencies, 
such as the Y.M.C.A., Bays' Clubs, settlements, Boy Scouts, 
camps, parks, recreation centers, and other agencies of -informal 
education Those in charge of the personnel services recognize 
two clear lines, of responsibility One is the function of select- 
ing and making sure that only those who are competent are 
admitted and certified to the professional ranks. The other is 
the function of cquneeling the individual in order that he will 
find the right road for himself and make progress toward bis 
professional goal. 


1 Discovering Candidates (14) 


Two- thirds of the suitable candidates for admission are dis- 
covered by present students and alumni of the college who as 
physical directors, Y.M.CJL secretaries, high-school teachers, 
and social workers are in close association with high-uhoal 
boys. Rather complete admission data are required— vix., 
high-echoal transcript, ratings by principal and others, scho- 
lastic aptitude test scores, medical history, and personal his- 
tory Each candidate is interviewed by at least one member 
of the Alumni Admissions Committee or the College Admis- 
sions Committee, persons who can judge the applicant as a 
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potential professional man Admission is baaed on intellectual 
competence, penbnal and social adjustment, leadership ability, 
health, physical qualifications, finances, and interest in a pro- 
fession for which the college educates, 

2, Continuous Counseling and Selection 

The college provides for continuous counseling services 
through the college course beginning with the orientation pro- 
gram of "Freshman Days" (4), which includes getting settled 
in the dormitory, registration, testing, social affaire, and expla- 
nations of procedures and traditions, and which u earned out 
to a large extent by a committee of upper-cliumen 

The Common Freshman Year (13) was begun In 1937 to 
provide general education to meet the needs of students In 
adjusting themselves to the college community and life's re- 
sponsibilities in a career of youth leadership 

A series of psychological and educational test# of abilities, 
achievements, adjustments, and interests (1) and a senes of 
physical testa of organic and physical efficiency, motor ability, 
posture, and nutrition (7) form the bans of a thorough pro- 
gram of selection and counseling throughout the student's 
career m college. 

In the group guidance course, "Introduction to Education, 
carrying nx semester hours credit for tho year, Individual re- 
sults of the tests are given to the students and interpreted as 
an aid to lelf-appreual and eelf-direction Along with these are 
considered the student's adjustment to college life, the meaning 
and purpose of higher education, methods of study, efficient 
reading, and vocational and educational planning. In the 
winter term, consideration is given to health adjustments and 
personal hygiene; and m the spring term, to religious, ethical 
and soclo-vocadonal adjustments. Against all these, the stu- 
dent checks his own qualifications and maps out: his curricular 
plans for the note year and the remainder of his college course. 
He makes application for admission to the professional division 
of hit choice and all admissions and freshraan-ydar records oF 
the student, including a profile of all aptitude, achievement, 
personality, and physical tests, accompany the application to 
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the professional division with the recommendation of the stu- 
dent's advisor and the Chairman of the Freshman year, Each 
freshman is interviewed by the Divisional Admissions Com- 
mittee and Is accepted or counseled as to dropping out of college 
or transferring to another more appropriate college. Thus tho 
orientation course, the testing program, end counseling not 
only form the bases of sound guidance to the student, but alio 
give many opportunities for objective appraisal of the student 
and careful selection for the professional divisions. 

Students enter the professional divisions at the beginning of 
the Sophomore year, Each man is assigned to a special coun- 
selor who helps him plan hla program for the next three years 
and supervises his work during the year. All men carry a full 
program of Liberal Arts and Science courses selected to serve 
preprofeseional values. Continued development In recreational 
skills is required At the end of the year, students participate 
in the College Sophomore Testing Program. The Sophomore 
year is a transition from general to profeisional education. 
Advisors and divisional officers using grades, test scores, and 
ratings, appraise carefully each men and make decisions re- 
garding him The promising men are encouraged to enntiniie, 
and the doubtful men are advised to plan other channels of 
endeavor 

The individual counseling by faculty members of the pith 
fusions I divisions continues ui the Junior year The course of 
study includes several professional courses, and two or three 
advanced courses in liberal subjects are required. Supervised 
field work begins In the local schools and aoeial agencies, which 
gives practical experience and tests the professional aptitude 
and Interests of the student In the Senior year, counseling by 
divisional advisors, usually by the professional divisional direc- 
tors themselves, continues. The proportion of professional 
study Increases and the program becomes almost entirely pro- 
fessional Student* can concentrate in their particular fields of 
Interest Field work increases and emphasizes experience in 
aupervision and is more critically appraised. 

Placement plans by die fogr-year men are made and creden- 
tials am filed in the college placement office. Correspondence 
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and interviews with employers are arranged The recommen- 
dation of men for particular jobi is the responsibility of the 
professional staff of each division. Final decisions and plans 
regarding the fifth-year program or other graduate study are 
made by those students who are qualified 

Those continuing for the fifth year increase their profes- 
sional study and field work and complete their theses or re-, 
search projects if they are candidates for the Master’s Degree, 
The same general program continues on this level with even 
closer faculty-student relations in all professional and educa- 
tional mature. 

Every student n expected to spend one or more summers 
in work experiences of educational significance. Most of the 
students become camp counselors, camp directors, playground 
and beach supervisors, or serve on the regular staff of some 
social agency which docs not have a speaal summer program 
Reports of these summer’s experiences are required 

The intimate social relations and physical conditions of 
dormitory life are influential factors in the development of 
student! Supervision of the dormitories is in the hands of 
a committee composed of faculty and students Upper-class 
counselors, chosen by the dormitory committee, counsel stu- 
dents concerning dormitory adjustments and personal ques- 
tions. Meetuigi of these counselors are held regularly, and a 
program of educating the counselors to more effective service 
Is carried on, 

Thu* throughout the college program, continuous selection 
and counseling procedures are in effect and make use of psycho- 
logical and physical measurements, 

3. Descriptive and Predictive Value j of Tests and 
Measurements 

Measurement is useful in making decisions ui selection and 
counseling because of its descriptive and predictive values 
The following pre-war data have been chosen as being more 
typical of the normal functioning of the college than data avail- 
able in the war years. The data in Table 1 give a picture of the 
average freshman student. These averages ire the result of 


and social wobk 237 

CMlfuHy In the opinion of the idminiltiMlon^tta' 1 ”’ 

SSXC 1 * type raMt ' U “‘ W ' - - EE. 

TABLE 1 


1 fasrsSs <U1 ’ d - —v >» 


1)2 


1- ii- is \ " i "’.h" 

l" 1 \ N I . 

■„* 1 ' — 
Pamuila (utwosl Bunns ) for Llbrnl Am Coll 
PnndpiTi Rabitp (9) 1 
Iuttamy , 

Sornl * * 


ion 


Sb 


jWsdalnlM Dib (1) * 

1 PotLclopal and Kduadoesl Tcsit— dus <u 

N Mutt 

Ouunsrin 131 ajj 

Tool nn m 77 _flo 

Hitters Compnktmion Tut. 

130 sutf 

1ml ,, J30 J«,» 

CnptretUit BnfUtk Tut (OM ) 

Total . . . ISO $0.40 

?,*eerel Culitm Tut 

8pdil Studio UI 3fi_2p 

gorafanLumnrs 131 jj/j 

f ™ Am Ill jo JJ 

Milium tin . Ill tan 


Vw 
6.19 

3.73 og 

SSI ]« 

in 

IJ7 


IS 


SJ) 

km) 

14JU 

>99 


Tut, Term C 


1120 


ftrmrib 

ilfemuICoOttt 

fnikutelfum) 

30 

a 

n 


J3J ?!■* 

131 I9J2 

- 331 136,65 

to . ISt fifi " hieh *■ B,, °* « “ 


Total 


M0 

1150 
13 JO 
1600 
1410 
gm 

3SJ0 


S 

60 


«t«d fmm 0 it ika imr ■ 


238 lOUCATTONAL AND PSYCHOLOQICAL MKASUlBMHNT 
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Tests of agility, large and imall muscle coordination, pos- 
ftire, and nutrition are alio administered during the first term. 
Analyau of data indicates that the freshmen who expect to 
major in health and physical education are superior to those 
majoring in the other professional divisions in general motor 
ability and capacity, In general strength development, and in 
motor development. 

A factor analysis cjf strength tests resulted in two com- 
ponents which can be k measured by a few simple teats, vifc, 
dynamic strength, which describes a composite motor ability 
criterion to the extent of 68%, and static dynamometrical 
strength, which was responsible for 24% of the reduction (8), , 

l ,/ T° r “‘‘ y * 1 * 1® cardio-vareular test variables and 

the McCurdy-Larson Test and the Tuttle Test isolated eight '• 
principal components, vix, three concerned with pulse rates, i 
two With blood pressures, two with pressures referring to pfl*- 
Cl™) C ^ ,an ® e * # * nc ^ one With circulatory recovery to expiri$& 

Other factor analysts of variables in specific sports, such as JJi 
soccer, basketball, and other skills have been made by VI 
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-.abmY (6), Tibbetts (12), Hunsidter (5), and othera. Im- 
thraratid and practical studies ot pwture and aquatics 
harebeen made by Cureton and of endurenca and nutrition by 

K *Kla2' aim aomo iHoa of th« predictive relations between 
tatscniM and achievement, Measurement haa been useful in 
pltdieuDS individual and group achiuvement 
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4. Effectiveness of Program 

The ultimate test of the validity of the whole selection, 
counseling, and academic program is its effectiveness in pre- 
psring men for youth leaderahip Pertinent to this subject are 
the data on academic mortality, placement, and professional 
stability given in Table 3. Data on Springfield College stu- 
dents for Part 1 of this table covered 0 yean, The initial place- 
ment data in Part 2 of the table covered approximately 900 
placements over 20 years Tho data on permanence of the field 
of initial placement were collected J> to 20 years after the initial 
placements involved. Strictly speaking, of course, a rigorous 
test of the effect of the program would require conducting a 
controlled expepment of such magnitude as to he impractical. 
However, it seems reasonable to believe that the relatively low 
academic mortality and the high placement and success in 
social work may be attributed m part to the use of measure- 
ment and prediction procedures in selection and counseling 
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GUIDANCE THROUGH SELF-APPRAISAL 

FORREST E. HEWITT 
Hum MBhirr Set«J 

It ia Well recognised that some guidance programs fail 
because the records become so elaborate and cumbersome that 
no one uses them. In the guidance program at Howe Militaiy 
School, this difficulty has been overcome by arranging the ma- 
terial so that a student’s problem can bo ascertained with from 
three to five minutes of careful study. The cumulative record 
system || based upon the recognition of the following three 
problem fields. 

(1) Self-guidance 

■ If today’" students are to be prepared to adjust themselves 
to this gadget-filled world of ours, it il moat necessary that they 
be encouraged to ifllf-guidance in the beginning of their high- 
uhoo) car«rs that they can become more self-reliant, can 
make better choice*, and will be better able to meet new aitu- 
atlons wisely This awakening to the realization that one must 
be able to guide himself Is one. of the moat important single 
factors of on individual's ea/ly experience. 

(2) Professional adjustment 

Too many high schools have allowed their senior* to leave 
school with the sole ambition of going to college If their parents 
could afford to send them. Many of these yoimgatcr* have, 
gone to college with no further atm chan to join certain fraterni- 
ties or soiflritiea, Education again was incidental and the pro- 
fessional careen were too far removed for immediate considera- 
tion. Too often these graduates have found that they took the 
wrong courses and would probably have been happier if they 
had chosen other fields of interest. These people too often 
sea 
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Interpretation and analysis. Through the assistance of the 
counselor, each boy wntes his oar* approval after drawing 
profiles of the various standardised testa , 

The fir«t of the senes of tests is a check on their technical 
information in the ten major fields, namelyi human relations, 
literature, government and history, biological science*, physical 
science*, machine shop, mathematics, business administration, 
office techniques, and architectural an On this same profile 
the student's high-school training and occupational preferences 
an charted according to national norma for the academic level. 

Each boy then takes a sonoa of interest tests in the aame ten 
major Adds and compare* the*e percentiles with hi* profile* in 
technical information, in occupational choice*, and hlgh-ichool 
training. These profile* are then compared with the vocational 
plan* the student has outlined in hi* autobiography. Each boy 
wntes a nimmaiy analysis of these profile* in the form of a letter 
of essay which n handed to the guidance counselor who inter- 
view* each boy in regard to hi* interpretation These letters are 
then given tn the English Maiterwho check* them forgrammati- 
calerroti and letter-writing farm. The corrected copies are then 
returned to the counselor who send* them to the bqy*' parents 
with personal letter* of explanation suggesting discussions be- 
tween parents and ion* during the spring vacation concerning 
the personal and vocational implication* M shown by the 
profilet 

At the first opportunity after hia return from vacation, the 
Cadet add* to hi* cumulative record in the guidance office the 
additional information gained m hur home discussion*. If it is 
, the boy's sen^r year, this in formation is used as a basis for 
in formation in hip future life plan. If he plan* to attend college 
the next year, this information becomes die basis for hi* Fresh- 
man college conferences and » placed in a folder and sent to the 
college qI his choice. 

Previous tq hu senior year, special aptitude teats are given 
to the Cadet and a complete inventory accumulated during the 
yean be attends Howe School Thu personal Inventory starts 
as rooo as the student begins his formal life in the classrooms 
pqd the barracks, sod accumulate* from the following sources. 
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become professional misfits and spend the rest of tW l™ 
wishing that they had chosen other career* 

(3) Self-appraisal 

Student, should end can be taught how to leem to , pp _i_ 
themselves, how to take advautage ol opportunity. form-tf, 
sod how to evaluate their progress. If s «If-« p l* nl ,~! 
manual consisting ol appraisal profiles sod check sheets u 
assembled and written for the student needs of each particular 
school, the guidance program will not become Ion m the dahv 
factual teaching. 3 

The lource material for the Howe record system is recorded 
in profile fotm on a manila folder in which the cumulative 
material is filed This information is compiled from the follow- 
ing sources 

(a) Technical information and penonal inventory tetti 
taken by the student. 

(b) Report* from the Master*, Tactical Officer*, the Head- 
master, and the Commandant, (These reports give a 
cross section of the work habits, conduct, and social 
adjustments of the boy.) 

(e) The Cadet's autobiography. 

(d) Counselor 1 * summary, 

Thu Hoto§ Profit System 

During the four days of the orientation period prior to the 
fall term, a complete check is made on each of the entering stu- 
dent’s achievement! In the fundamentals of reading, language, 
mathematics, science, and the social studies. This battery n 
scored and placed on Individual profiles showing the student’s 
actual status m the tools of learning Thu information U amt 
to the Masters who begin their "special help" periods for the 
new student with specified difficulties 

The entire program centers around the building of profiles 
showing achievement*, interests, personality, and social adjust- 
ment. At various times during the school year all of the bpyi 
take ■ senes of interest, personality, and special ability tests, 
the rcores of which are placed on profile* by the Cadets for later 
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(1) Profile* of the boy's daily scholastic work which ate 
chocked by each Master under whom he is taking 
courses. 

(2) Tactical Officers' report* showing a picture of the boy’s 
adjustments to life in the barracks 

(3) Headmaster's profiles showing ratings of respect for 
authority, leadership, ability and willingness to co- 
operate. 

(4) Commandant's report summarizing punishments and 
reaction* 

Sinee the rating on many of these factor* must of neceialty 
be by subjective judgment, then profiles on the student's work 
habits, emotional control, coopenPon, leadership qualities, and 
Future plan* are arrived at from various source*, thus giving 
a more reliable evaluation This summary profile in the form 
of A penonal adjustment inventory has proved invaluable for 
counseling purposes and is placed in the Cadet** cumulative 
record file in the guidance office. 

After the analysis of the profiles and inventories, there 11 no 
attempt to pigeon-hole the student Into a given profession w 
particular college. Instead, wise counseling Is combined with 
student freedom of choice The selection of professions Is often 
born in the bi-monthly meetings of the College Preparatory 
Club For the under-classmen the broad choice of whether the 
boy want* to work with Ideas, thing*, or people' very often 
come* from hit study of the career data and charts in the 
guidance office. This selection of a career by including the boy 
in the program from its inception gives the boy (he feeling of 
responsibility and security, as he knows that he ho? some defi- 
nite and specific plans for his future which can be changed, if 
necessary, to meet new situation*. 

Vs* oj the Autobiography 

In order to help the high-school student to know hirnJf, 
the autobiographical method has preyed indispensable. 0t* 
senior entering the school for the firet time lost fall, itotp& W' 
the clow of hi* autobiography, **| *dll don't know exactly 
I want to do, but; writing thia ha* helped me to know mjr*^ 
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better, and I think by the close of the Hemeacor I will be able to 
pite an intelligent decision, ” 

Danng his Junior year, another boy decided that he wanted 
to become a chemical engineer, but when he and the counselor 
Y'rillied b, s aptitude and achievement profiles, he was made 
„Mie of a definite weakness in mathematics (The author 
yi-Ycl that one of the most effective features in this technique 

that it offers tho counselor the opportunity to assist the atu- 
fat in discovering his own interests and deficiencies.) 

When writing his autobiography, this boy stated that his 
father was too busy to discuss his personal and vocational prob- 
lopwlth him. Hw counselor suggested that if the boy prc- 
uited a definite plan to his father, he was sure that hu busy 
tidier would take time to advise him This Junior not only 
*ttip a definite program, but also followed the counselors aug- 
jationi to tiy out different jobs during Ins vacation. 

The broadening experience of this boy through the explora- 
fojy jobs, and the new rapport developed between father and 
nn brought forth the proud statement from the boy when he 
gamed to Howe fur hw Senior year, "Now I know my dad, 
tad yrt decided that it would be better for me to change my 
erntr from engineering to dentistry.” He went on to explain 
aim revised autobiography his reasons for changing hw course, 
ad gave in detail his new plans. 

Autobiographies are written during the Frbshman year and 
prised sach consecutive year They are considered absolutely 
ctffidsntial and are filed with other cumulative material in 
each student's guidance folder In the counselor's office. Certain 
puts of the following autobiographical outline are revised and 
brought U P t0 date from time to time. 

I life hutoTy 1 

fa) Ancestry 

(b) Description of homo life 

(c) What tha boy thinks the parents expect of him 
II. Difficulties 

(a) School 
(b Home 
(c Social 
(d physical 

(e Any other personal problems 
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HI Achievements 

(a) Records, medalt, etc. 

(b i Sports 

jej Jobs held 

(d j Money earned and money saved 

IV Intsreats, hobbies, and activities 

fa ) Description of interest in hobblea 
(bi Amusements preferred 
fc> Reading interests 
(d) Travel 
V, Emotional needs 

(O Personal and aoaal adjustment in school 

(b) Religious 

VI Lue clans 

(* J Professional Intercna 

(b ) Vocational experiences 

(c) Analyiu of abilities in relationahip to chosen field of 

occupation 

___ _ (<0 College and technical school plans (Seniors) 

VII Summary 

A short interpretation of the implications of the tat 
results in companion with the auttmlography and future 
plans as previously made with parents 

Each autobiography is read carefully by tho counselor, and 
if it is incomplete or not well written, tho paper must be revised 
and rewritten. 

Practical OMtconuu 

This type of cumulative record offers the counselor both 
tangible and objective facts Tor interviewing each student 
These records also help the student to decide upon more defi- 
nite future plana, and offer an incentive for him to do much 
better school work. The profile system gtves needed motiva- 
tion for the superior as well os tho average and poor students 
who have previously failed to work up to their ability levels 
These objective records also bring to the students the impor- 
tance of personality development and its effect upon their social 
and school life. 

In meeting the needs of each student as an individual by 
teaching him how to study, how to choose snd prepare for a 
vocation, how to get along with others, and how to interpret hu 
experiences which will enable film to solve present and future 
problems, the plan is helping the boys to adjust themselves to 
the rapidly changing world into which they are being graduated. 
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TRIAL AT YALE UNIVERSITY OF THE ARMED 
FORCES INSTITUTE GENERAL EDUCA- 
TIONAL DEVELOPMENT TESTS 

ALBERT B. CRAWTORD 
an 

PAUL B BURNHAM 
Yile Unreality 

Among several major plxtblemi rclaDve to demobllizaticm 
with which our colleges are Immediately confronted u that of 
evaluating, fairly and with reasonable assurance, the scholastic 
promise of individuals now serving the Armed Forces Edu- 
cational provisions of the "G.I Bill" in due courae undoubtedly 
will bring to the colleges a flood of applicants for admission. 
Among these will be many "irregulars 1 '— those with broken or 
deferred educational histories In the formal sense, yet with valu- 
able training or new skills acquired In service Since these 
personal developments will be difficult, if not impossible, to 
asaay in traditional comage of the academic world ("units” 
and "credits"), colleges at last may be impelled to consider 
what the prospective student knows, and what educational 
aptitudes he can demonstrate, regardless of where or how these 
attributes have been obtained. 

The Armed Voices Institute has made educational programs 
of extensive scope available throughout all major theatres of 
training and operation Certificates 'attesting satisfactory 
completion of particular courses, or achievement-test scores in 
specific fields, are issued through “GJJ.Q-" for this purpose, at 
die University of Wisconsin, Madison, However, the Armed 
Forces Institute measure mbst useful to colleges in their selec- 
tion of future, demobilized students appears to be the General 
&4vcaiional Development Battery of four tests, viz* 

Ml 
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Test 1) Correctness and Effectiveness of Expression 
Test 2) Interpretation of Reading Materials in the 
Social Sciences 

Test 3) Interpretation of Reading Materials tn the 
Natural Sciences 

Teat 4) Interpretation of Literary Materials 


No further discussion of these testa is necessary here, since 
they have been widely publicized and are fully described id the 
Examiners* Manual issued by the American Council on Edu- 
cation. 1 This Manual contains valuable remarks concerning 
the nature and purposes of the General Educational Develop- 
ment Battery, and present! tentative cqlloge norms for each 
test. It stresses the need for local norms as well, stating. 

These norms ire intended tn hdp the schools dead* what mini- 
mum teat perfonnance should entitle returning wiMtivia pemms to 
t given amount of aademic audit in a given ares. Became of the 
heterogeneity just noted, however, there il danger chat certain schools, 
by uncriDcsf acceptance of the general norms here reported, may Kt 
standards inconuitent with their local needs It it highly desirable, 
therefore, that each widiudoa, by administering the avilisn forms to 
its own dvflisn students establish its own local norms for use along 
with the genera] norms in the interpretation of scores of service nun 
and woman reported to the institution by the United States Armed 
Forces Institute. 


To that end, particularly as jt might affect the program of 
Yale Studies for Returning Sendee Men, an experiment waa 
recently conducted at Yale University All members of the 
civilian freshman class which had matriculated in July 1944 
were Invited to participate; 135, or about one-third of the total, 
dfd ml Although the invitation put no pressure upon fresh- 
men— since only interested volunteers were desired — it stated: 
a) that parddpatlon would be of direct aemce to the Univer- 
sity and collaterally to men now in the Armed Forces, b) that 
reasonable compensation would be given for the rime expended. 


1 For further Wonrutwa 
laitnut* wcClid Vm-I 
Mi, Tuii d Cntn! 


onnstiaa OB tka Birury and «h*r sham ef tin Anu4 Form 
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For each of these four ceit* comprising the A.F.I General Edur 
cMlwnd Development Ball try' two hours are allowed. 

In appraising their performance, it id essential to note hovr 
well the tut group represented, in scholastic promise and 
achievement, the entire class. Fortunately it was found that 
the random sample of participants in this experiment covered 
virtually the entire class range, from highest to lowest, in aca- 
demic standing, for the first complete term of Freshman Year. 
Average score oi this group on the College Entrance Examina- 
tion Board's Scholastic Aptitude Test was 575 and on other 

TABLE 1 

ATI Tail Group Comfarai trill Total FraiAman Cku of JW7 


CEEB Verbal Bant* 
l* "iu V 'W' 'V-J« 

4.V ■ ' « 


?d 


84 


• Imjnjirido In icaxxJ«ry-«djool record alma mi kWiiIjc prafietjoa* fa tom* 
mo) ■Jimnitwo, ctikfly lor milory lertlfo, nirsni ireriaun matricuhnti latunfl? 
muoa lor thfl d [Strewn between 453 iiul 411 In tout daw numben repn*j*it«3 
above, i 

tests of the College Board or the Yale Educational Aptitude 
Battery of the same order— sroilnd 20 standard deviation 
abovfl the dans moan Hence these voluntary participants 
ranked, in pre-matriculation indices and later scholastic 
achievement somewhat above the median of Yale freshmen*, 
i e i around percentile 60 rather than 50 in academic promise 
and accomplishment. Moreover, despite relatively high aver- 
age performance of this freshman group on the A F.I tuts, 
dispersion of their scores followed a normal-probability pattern 

/The Barnxy smptoed U dur obtunihU threw th the Coependvr: Tat Service, 
15 ABMunlim Avan*,, New York Qty — l pin lid Iona re ifast icttully mat by the 


Time Allowances 

Maximum tun. tllowoil far e!ell telt . 
indicated in Table 2, many Yale studenta dnulijJ 
ably leaa time, The low correlation! there ihown llio !?"’ 
that lanjth of „m= spent or each taa, bora IMet“. Si" 
to score made by these freshmen. The time 
generous and the testa (a. intended) therefore Kern to fej . a 
on 'power” rather than on speed ” ,w 

TABLE 2 
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AFI Ten n (Sochi Studio) , 
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65 nje. 

75 pint. 

BQ DRL 

93 rain. 
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Since ell four ten of thi. batteiy. Including that designated 
as Interpretation of Reading Materials m the Natural Sci- 
encct* are highly verbal, a considerable degree of positive inter- 
correlation among them would be expected The Exam**/ 
Manual rather surprisingly offers no data m this respect. The 
following table presents these, for the Yale freshman tut group. 
The correlations in this and subsequent tables were obtained 
from standard scores for the AFI teats, derived from raw score* 
by uso of the conversion tables furnished in the Exmmtrf 
Manual. 

TABLES 

InUrtemUltoni emote ATI TVrfr (Jf= IJS) 

_ nr If TFT 

“ "*■—■> sis 


I (Ere rot fan) 


This correlation matrix indicate* rather clearly (on the 
basis of a email but representative Yale sample) that the 
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Armed Forces Institute General Educational Development bat- 
tety is general, rather than specific or differential, throughout 
its several parts. We hasten to add that no derogation is im- 
plied by the foregoing statement. It is simply not in the na- 
ture of this Battery, nor In the aims of its distinguished pon- 
itruccon, to measure differential rather than general scholastic 
aptitudes; which indeed should he clear from its title and from 
objectives discussed in the Examiners’ Manual Noverthclui, 
th«w coefficients ate not so high as to suggest that all four testa 
measure the same mental functions. It may be that similarity 
in form (i e», the faet that all four are highly verbal) accounts 
Tor the intercorrelations, and that dissimilarity in content 
accounts for their not being even higher 

Companion of Armed Forcer Institute wtk College 
Entrance Examination Board Tests 
Although the content and objective* of A J.I. General Edu- 
cational Development Tests differ considerably from those of 
College Entrance Examination Board measures, certain ele- 
ments of the two teriu, from Yale evidence, appear functionally 
tq have much jn common. A chief difference of objectives 
concern* recency of fonnal schooling; Le , College Board ex- 
imi nations are meant to be taken in stride by students n taring 
completion of their college-preparatoiy work and therefore to 
afaubsuntial degree quite properly measure current, specific 
achievements. TheA.FI educational development' battery, 
noTets properly, attempts to measure combined aptitude and 
achievement (however acquired) in -more general terms. Yet 
the following ratio indicate* a rather high degree or corre- 
sjtoudence among those parts of each test-senes which by tide 
seemingly represent analogous educational fields. The Yale 
experiment affords direct comparison in this respect of the two 
testing methods. Candidates for admission to Yale are nor- 
mally required to tiled the Cbllege Board Scholastic Aptitude 
Tert (both verbal and mathematical sections) and the Englvk 
Essay Examination, “They distribute themselves rather widely 
Wkwgdipijthw Collage Board options, according to subjects 
P* W in the ainior-high-school ywr and Intended college 
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program. Consequently some of the correlations represented 
below, and others in Table 3, are based upon too small a num- 
ber of cases to warrant general conclusions By way of cam- 
paruon, firat-ierm vereut second-term grades in the Freshman 
Year (Class of 1945W) correlated: English 10, .61, History Iff, 
73, Chemiatiy 11 and 14, _S7; Physics 10, 76. 

Despite the numerical limitations of there data In some 
categories, it appears evident that the Aimed Forces Institute 
testa correlate surpruuigly well with those of the College En- 
trance Examination Board In related areas — especially when 
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allowance is made for the high average performance and there- 
fore somewhat reduced "spread" of Yale freshman on the A.FX 
Batteiy. i 

Relationship of Scores on AJFJ, General Educational Develop 
mmt Battery and Certain Goffs gt Board Tests 
to Fvst-T erm Grades at Yale 


A pragmatic and customary, though by no mum ideal, 
method of evaluating teats id to correlate scorei thereon with 
s ubseq uent grades in course. The latter, however carefully 
assigned, because of their subjective nature have dubious «*- 
tUticnl reliability. However, m many situations thqy ofer' ^ 
the fairest criterion by which the relative value of different ! 
prognostic measures may be compared. The next exhibit 1 ' 




/T.kli 5) orsuiu ■ nenefl of correlations between test ecorea 
SL-tertn pair., of participants m thli experiment 
It I* meopiieeif tint reverel factor! bearing upon compin- 
. a, College Entrance Examinetion Board teata and 
Uelltf Omral Educational DmlopmtM Tuts in Tablet 
JfriS n..yXt the rerulte there given. All US men took 

TABLE i 

D»U» al 

(Dili bud ofl A FI Tut Group of U5 F rubra ta) 


^ A«W ” 

CESB EojTaii ferny . . 

M (An Cm) 4H.U 


CEEHSpitul 3J 

«S JBWSSi--“:i i 

■ iS jS 

•ells iwAmp .« & 

1 CB 8 B Gmnl*Anjnre .... . 44 J» 

Avnnp e4 ill CEEB mu . J3 U5 

Mwim d CEEB SoiM Suidfa* ud £>1^ tml*. 

the nme A.FI tests but alUdjd not taka the ame College 
Bnid tats, They all did take three of the latter, Scholastic 
Attitude Test (Verbal), Mathematical Aptitude and English 
Iriaji. The remaining two College Board teati taken by each 
oxidate for admuaion are election! determined by his secon- 
duyachool couraa and prospective college major and are drawn 


from the following options: Social Studies, French, German, 
Spanish, or Latin Reading; Biology, Chemistry, or Phones; 
Spatial Relations ( Three-Dimensional) Visualising Conie- 
quendy the average on College Board Examination! is not 
based upon an identical battery For all men represented in this 
study, Each candidate's options, however, are appropriate to 

his expected freshman progranj. 

Most of the students represented took their College Board 
tests in Apnl, 1944, whereas the A F I. battery was admin- 
istered near the end of the fint term. Consequently, the in- 
terval between taking College Board testa and obtaining the 
final grades for the term (silt months) was considerably greater 
than that between A FI. tens and the and of term (one 
month) There is no way of estimating to what extent these 
differing time intervals may have affected correlations or the 
respective tests scores with the criteria. 

Again the number of cases upon which some of these con 
relations are based permits only tentative conclusions; It does 
appear, however, that 

1) A.F.I. tests show promise of being wholly acceptable 
alternates for College Board examinations in the verbal sub- 
jects. 

2) For this sample group, A.FI Total Score correlated as 
well with Freshman first-term average in all courses, as did the 
average of all College Board tests, 

3) The A F.I General Educational Development Battery 
makes no pretense of measuring abilities in Mathematics, Me- 
chanical Drawing, or Descriptive Geometry. The College 
Board MA T, and Spatial Relations tests are probably indis- 
pensable for prospective scientific or engineering major*. We 
have obtained no information regarding A.F.1. teati in specific 
branches of mathematics, but these would have pertinence 
chiefly for students who hays completed those particular 
courses. 

4) Since College Board MJL*T scores are likewise some- 
what dependent upon previous mathematical training, inter- 
ruption thereof might adversely affect individual performance 
on this teat also. By its nature, the Spatial Relations score is 
leas likely to be so affected. 
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From joint inspection of the A.F I. teat scores and freshman 
grade distributions, it was a relatively simple matter to de- 
tsmune two critical-score levels Practically no students scon 
mg above the upper one had unsatisfactory first-term records, 
while most of those scoring below the other ranked well under 
tht class average. As a result of this investigation it was voted 
hy the Executive Committee of Yale Studies for Returning Son 
vice Men to admit candidates scoring above the higher critical 
level on the A F.I. Battery despite their deficiency in formal 
academic 'credits, to reject or discourage (unless additional evi- 
dence more favorable to their chances of success' is submitted) 
thode scoring below the lower critical level, and to examine by 
other means border-line cues falling between these upper and 
lower levels. 

For the reasons noted above in Conclusion (3) further teats 
(of the aptitude rather than formal achievement type) will he 
required of prospective engineering, mathematics or physical 
science majors. 
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THE CRITERION 

HEIBERT A. TOOPS 
The Ghh Sum Umvtrelcy 

Ih all tot work, and m malting prediction# generally, there 
must he— according to the current mode of the thought and 
Its consequent development into corresponding formulas— a 
unitary, general success score, or criterion score, Cor each person 
or the experimental group by whose aid the tests are con- 
structed, combined, and validated 

Item analysis lor selecting the better items requires such a 
criterion. So does item alternative analysis, dune with tho 
purpose of altering, in the hope of improving, those confusion 
alternatives which are "out of lino,” which have, for example, 
positive item-alternative validity coefficients instead of the 
muchrto-be-derirtd negative ones. For a multiple-choice test 
we desire such alternatives— for right answer and confusion 
alternatives— that the validity of each item may be as high as 
possible; or, to put it in another fashion, that m many as possi- 
ble or the experimental items shall have a validity coefficient 
above some arbitrary lower limit, say M for testa of intelligence 
at the university freshmen level In the former case, which 
presupposes tho second having already been satisfactorily ac- 
complished, we are concerned with the picking of the individual 
items such that, for the n item* chosen for a given teat, the cor- 
relation of the sum of the scores (the Rights score) on the 
items-as-a-whok (l-e., the test as finally constituted) and the 
criterion shall be a maximum IF the intereorrelations of the 
items are low, approximating zero, we may be content to ignore 
them; hut we can hardly proceed at all without validity coeffi- 
cients, or some reasonably adequate substitute therefor, the 
indices or which of necessity alio require a criterion for their 
computation. 

an 
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Having arrived at * "pretty good" items, each of deniable 
alternatives, we may hope, if the test is a time-linm ten m 
improve further the validity of each sub-test by utilizing 
lug formulas, which, if we are to use them, "must duco**. 
emus” (ie, C must be negative in S-JJ + C IF). Thu too 
presupposes a criterion because the weight of Error*. ^ 
Wrongs, (IF), i e., errors (5) plus omissions (0) relative* to 
Rights, can be ascertained (as vs. judged arbitrarily) only by 
the aid of a criterion and the multiple regression equation Tbn 
selection of tests for a battoty presupposes that cpnmdtnUa 
numbers of sub-tests are tned out against the criterion as a H tt 
of ultimate measuring rod, and that only "the few best" lrc 
chosen 

, To weight these chosen tests in a multiple regression equ>- 
tion then requires also a criterion The purpose here u to n 
weight the several sub-tests that a maximum validity of the 
composite or "scale score" results Even if the weights are irin- 
tranly chosen a criterion is needed in order to compute a 
validity coefficient of the thus arbitrarily weighted scale. 

Finally, then, we may say that we require a criterion for the 
general purpose of maximizing or optimalizing all sons of reh. 
uonships in the problem of prediction; for determining che poor 
alternatives and choosing the better items, for determining the 
best scoring formula, for selecting the better sub-tests, for de- 
ciding the weights to be ascribed to each in a battery or "scalo" 
for determining the validity coefficients of batteries or scales 
qf tests weighted either arbitrarily or by multiple regression 
equations, for determining the "most causal" of all the predic- 
tive variables (inclusive of traits and environmental facton or 
variables and social relationships) and determining their rela- 
tive weights; and for determining what basic form of relation- 
ship (including therein "higher forma of the multiple regression 
formula") underlies the beat combination of all the "causyd” 
variables For all such purposes a unitary criterion score for 
each person of our experimental population is indispensable. 

Yet success is nos unitary! A concrete consideration of 
cases will emphasize the point, Miss A », let us say, +1 o in 
speed-cf- typing, but is -1 o m accuracy-of-typing (that la, she 
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"is speedy but inaccurate") . Miss B Is - 1 o in speed-of-typing 
but u + 1 a in accuracy-of-typmg (and popularly, "is slow but 
accurate"). If for each of these persons we add the scores, or 
avenge the scores, or (for these particular two persons) weight 
the scores equally, in oil three circumstances they turn out to 
be 0 o on the composito-of-specd-ond-accuracy variable, or suc- 
cess-ao-a-typwt variable; that is to say, both are "mediocre." 
Yet by no reasonable stretch of the imagination are MUs A and 
Miss B equally "successful” typists The more true statement 
is to say chat they have a different kind, or a Afferent type or 
a different pattern of success. Most employers of stenogra- 
phers will prefer for most purposes Miss B, because they hate 
errors, they hate the correction of mistakes and the complaints 
of customers and clients which inevitably result when errors are 
made. And besides, even if Miss B does leu work, still very 
little of It has to be done oyer again. And doing work over 
rakes much time— much more than enough to do it correetly 
in the firet place— and costs money. 1 

Mire A and Miss B do have a different profile of typing suc- 
cess. Thu suggests that, possibly, if we could hold them to the 
■amespeed of production (that u, keep speed a constant) then 
their relative errors might measure their relative success, or 
conversely, if we could experimentally keep their errors a con- 
stant their resultant speed would yield us a unitary measure of 
their success; tbit is, that in either of the two alternatives we 
should have not two aspects of success but rather only one; 
and instead of having for each a two-traited profile we would 
hsyc a unitary success score such ai our ordinaiy regression 
and weighting equations demand. 

A little thought will reveal the. fact that thia u not possible, 
psychologically, for even these ta o variables; for hold down the 
•peed of Miss A and instead of becoming more accurate (as 
almost everyone presupposes will be the result) the converse 
actually may occur; while speed up Mire B and her accuracy 
in all probability will improve; at lean the normal expectation 
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la that, after a minimum of practice, such will occur. It all 
depends) 

Also, speed and accuracy art not the only variables m suc- 
cess. Success is not a two-traited variable compounded only 
of speed apd accuracy. If Miss A and Mire B are private secre- 
taries their employer considers as aspect! of their mem or worth 
other abilities as well; how wall, for example, they can spell, to 
cover up his ineptnere in this respect, the excellence of their 
grammar, so that they never asy in a letter "have went” even 
though that may be what he dictated into the dictating ma- 
chine; their ability and adroitness in answering the question! 
of visitors and in answering the telephone, and even their 
ability pleasantly and skillfully to entertain an influential cus- 
tomer until the boss's return In other words, even in simple 
jobs success is multi-dimensional Or, finally, w> conclude that 
success on even a simple job is measured by the individual 1 ! 
profile on a multi-dimensional profile system composed of » 
criterion variables, where, generally speaking; m is greater 
than 1. 

The upshot of all the argument to date, then, would seem 
to he that for an adequate treatment of the prognostic problem 
we should strive to predict the individual's success-profile, 
rather than to predict any unitary combination — however skill- 
folly combined — of those criterion variables. 

Thu prediction of a profile, n vt a unitary criterion scots, 
may be done by the simple expedients of: 

1 Administering enough testa of a wide enough variety to 
predict well (a relative torml) each of theaevenl cub portion 
of a criterion — the wage*, the foreman’s ratmga of indiip* nn ~ 
bility, the accuracy and the quantity of production over com- 
parable times, and so on 

2. Finding all the possible in rer-correla turns, 

(m+n) (m+n+1) 

2 

in number, of them criterion variables and the *t test variables. , 

3- Employing each of the m eub-enunon variables in turn „ 
as s criterion and determining the appropriate regression eqijto \ 
tion for Its prediction by the aid of the n several tot vaiuwsk,* 
m each case. 
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. Making fft different Y predictions for each examinee by 
of the i» thus determined regression equations! and 
"Sduenthr plotting for him these 7 * predicted criterion scores 

““1*1. on « regular profile chart, the plotted profile thru 
M ifie predicted profile of the person n queetlon > 

Such e profile, in common with all profiles, reveale vuually 
a rebel in general three things , 

1 The general trend, or tenor, of the scores. 
i The strong points and the weak points of the individual's 
■ereral criterion "abilities ” 

3 The dispersion of the criterion '‘traits’ about their cen- 
tral trend .... , . . - 

To the extent to which the several sub-portions of a cn- 
tenffli art highly related (possess a common G, or mathemati- 
a \ factor) will the standing in the several traits be more or 
fo, identical and tend, consequently, to result in a senes of 
ntings Which are located in a horizontal lino The norm a ted 

tqtrS (the encircled "ovals") of all columns of the plotted 
profile would be identical if all si/eh predicted eub^ntem inter- 
correlated perfectly. ’ . » . 

The mode of using such profiles — in selection, for example — 
fa not generally agreed upon Consequently we are concerned 
tore often than not with combining the several sub-criterion 
nibbles into a unitary criterion score, for Jo chat case the 
ample concept of a “critical score” applies. 

'Hio success, then, of an individual, about which we prate 
10 glibly, is a complex thing, and if it is to be made, artificially, 
mm a unitary variable must be compounded of the weighted 1 
mm of the several component parts, as one, simplest, concep- 
tion of the matter. If we accept that definition the problems 
become three in number* 

«*I vari»Ut, y u wffl hm * itandatd devfarfao, Or t = 
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1 To decide what variables are to be included in the cri- 
terion* 1 1 , to decide what variables are to be weighted other 
than zero 1 

2 To decide In what units— comparable measures— to 
record the individual's standing in the its several sub-variables, 
or component Variables, of the criterion, Unless the several 
sub-portions are recorded m comparable scores, the multipliers 
of the scores will weight the variables other than intended 

3. To determine how these are to be weighted, i.e,, to deter- 
mine by what specific weights we shall multiply in turn the 
several sub-entenon scores of John Jones before adding the 
several resultant products to obtain his final “weighted” cri- 
terion score. Let us consider each of the three problems in 
turn. 

The Sub-Criterion Success Variables 

Depending on the problem at hand one will have different 
success scores. Thus m a study of divorce, one would consider 
for inclusion in a criterion such variables aa length of time that 
the marriage existed before the divorce, the happiness scores of 
the husband and wife at comparable times, or the sum Or aver- 
aga of the two as an index or measure of the ‘‘happiness of tha 
marriage,” the number (and the quality) of the children in 
which the marriage resulted before tha divorce (if one is inter- 
ested in the qugenic&l aspects of divorce) and the like. The 
above statement suggests three corollaries. 

1 That the purpose of the study, whether legal, moral, 
social, eugenical and the like, will in part determine, if not the 
individual variables, at least the fields or realms in which we 
may look for the variables to be included. 

2. That there are always variables which are on the border, 
land of “test” and “criteria”; ones which in one study are on 
one side of the equality sign (of the criterion -weigh ting equa- 
tion) and in another are on the other Thus in a study of 
divorce m which the length of the marriage (stability or per- 
sistence of the marriage) u the focus of interest, the number of 
children logically Is thought of as being a "test,” the presump- 
tion (or hyppthesis to be tested) being that where there are 
children a marriage probably is more stable or lasts longer than 


THX CRITERION 277 

where there are none, (There could be curvilinear relationship 
between number-of-children (X) end the persistence in yean 
(F) of the marriage ) ' In another study where the center of 
interest is the social productiveness of the marriage, such as the 
number of children, the quality of the home, the wealth, happi- 
ness, inventions and personal adjustments secured, the number 
of children is certainly one of a mipiber of alternative “prod- 
ucti” or the mamages studied and quite dearly, then, is one 
of a number of alternative criterion sub-variables. 

3 That there is no universal agreement as to what consti- 
tutes ''success" ui even one realm, not to speak of it aa a gen- 
eralized measure m all realms of life. Concretely, we do not 
agree even as to “why, or for what traits of success we pay a 
given man one wage and another another.” Obviously, then, 
to «y that “Mr. Jones Is a successful man" means nothing defi- 
nite statistically, even if, by common agreement, it does mean 
that the man in question has been successful in a financial way, 
probably that he has secured promotions, recognition and even 
acclaim; and possibly, though not surely, that he has lived a 
good and useful life — in other words, it suggests much but says 
nothing definite and positive Since there are no universally 
agreed upon definitions of success nor even concurrence aa to 
what variable* to include, we must be aibitrmy in any case, 
Wt may seek to militate the possible had effects of our outn 
judgment by pooling tha judgments of other (competent) 
people of to tehat variables to include . ^ 

For our purposes here jt will be necessary to restrict the 
discussion to some one concrete realm. The one of greatest 
development to date is the vocational More criterion varia- 
bles have been devised in this realm, probably, than in all 
others combined, Let us restrict our attention then to it. 

In this realm, then, we may ask ourselves what are the 
evidences of vocational success Let u* enumerate some of 
those which have been employed In such studies, while in order 
to save space we discuss each briefly at the point of mention 

1 Wages, In free competition, where workers are all 1 doing 

the same kind of work) and promotions “ou the b|Ml« of merit” 
are promptly granted, wages evidently pre a fair tp good, If not 
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excellent, aspect of a man's worth. A little reflection, how- 
ever, will reveal that wages often are paid for quite other 
reasons than “merit displayed in free competition.” One man 
is paid more chon another of conceptually equal merit became 
he ia a relative of the boaa; because he ha* more dependent! and 
“Deeds it”; because he aaka for raises more persistently; because 
the boas thinks well of him, or simply “likes him”, because the 
trade union to whieh he belongs has a minimum wage scale for 
persons of his "experience”; because he has been employed 
longer, etc., etc. Conversely two men of conceptually unequal 
merit may be paid the same wage because the company pays 
only a few different "day rates”; or because promotions come 
only at stated intervals and wages generally are meet highly 
out-of-line Immediately preceding a "promotion period”, and 
the like. 

Again wages often nowadays are paid according to a "wage 
formula” which “recognized* many other factore besides basic 
“productivity.” That fa to say a wage, or a part of a wage, 
may be a bonus intended os much to motivate die person along 
certain lines of desired behavior as to reward him for “merit 
already displayed ” Thus card mess or absence from work may 
be penalized mote heavily than the hours or nunutes lost would 
warrant In the hope of inculcating “punctuality” and "depend- 

ability” 

In general, equal mages stall retied equal meat only mhen 
tha attendant circumstances are equal, where, for example, the 
number of days or hour* is equal; and, Jo general, where the 
*risk/‘ ore equal Where the wages are equal hut the risk 
unequal, equal wage is not an adequate measure of die relative 
worth of the two men receiving it. Concretely, m the first of 
two sales territories ic may be harder to sail goods than in 
another (of different “need,” or demand, and ■odo- cc pnonue 
, circumstance). Consequently equal commipaions. received in 
the two mentioned would indicate greater "sales ability" on the 
part of the man ht the "more difficult” territory, The problem 
of sayjng precisely how much better is not necessarily eefived 
by an arbitrarily devised correction, , 
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Absences (com work, if not allowed for, make the ‘'pay 
envelope 11 an erroneous reflection of the individual’s merit.* 
Anything which arbitrarily reduces the standard deviation 
of the resultant sub-criterion Korea has a deleterious, or attenu- 
ating effect on the resulting Korea. Thus, in an effort to pre- 
vent "speed up” and consequent possible reduction of wages 
for doing a given amount of work, wnrkara under bonus sys- 
tems often stereotype the output; that is to say, all or all but 
a few do on any one day practically the same amount of work 
despite their ability in individual cases— possibly in a majority 
of cases— to produce much more Although the record of a 
day, or any number of days individually may show this, the 
swnmated record of a week or of a month may not he revelatory 
of this situation Under such conditions obviously the produc- 
tion u all but meaningless. The correlations with production 
of any truly good prognostic tests in this case will be attenu- 
ated, that Is, greatly lowered Where uniform or practically 
uniform day wages are paid, soldiering on the job on the part 
of die more capable workers may have die same end result on 
any meamiiw of quantity of production which may be collected. 

Where workers work In pairs, aiding each other in simul- 
taneous operations, the speed of the one will of necessity ha the 
speed of tho other, as in all "araembly line" production and any 
criterion, other than “errors,'’ becomes largely meaolnglejs 
The respective wages in this caw are a measure of the "wage 
formulas" employed and not of individual differencw of merit. 

And one may be paid for “power to achieve" rather than 
for "achievement," because, for example, one can do hard jobs, 
or rare jobs or specialized jobs which another of tho same labor 
force cannot. Even though this month both do the same work 
the fint of two men may be pud more because he, alone of the 
two, could do the unusual job if one came up, diet is, the em- 
ployer is willing to, and does, pay for insurance that h« finiv 
“can handle all jobs that may coma its way.” 

Often also there are notorious buses m wages. In Brasil 
it u said to be the custom for a worker to ask for, and- confi- 
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dently to expect, a raise with the advent of every baby in the 
family, and "ability” in this respect by no means is perfectly 
correlated with job-worth Again, women leathers, it is com- 
monly believed, often receive far less wages for comparable ser- 
vices rendered than men, the theoiy, or justification, possibly 
being that “men support families" while "women work for pm 
money” and should be paid accordingly. And of cou TC t the old 
law of supply and demand still obtains! In such pares one may 
divide the investigation into two independent studies bared on 
sex" 


2. Production. Where all the workers of the experimental 
group are working at “exactly the same work," the quantity ol 
work produced is a good measure of the merit of the individual 
men Production in general is what occupations are for. Yet 
current production, even on excellently planned "bonus work " 
may reflect only, or largely, irrelevant factors such as a man’s 
temporary need for money, as for paying off a mortgage soon 
to come due, or reflect too largely his current condition as to 
sickness, fatigue, morale, or interest 

And, critical inquiry often reveals that the conditions of 
work arc not equal, that, for instance. 

One salesman has odd jobs to do which subtract from the 
tune in which he can give his full attention to lus main job of 
selling. 

One salesgirl, as versus another, is required to sell a higher 
proportion of obsolete stock, which consequently sells more 
slowly than the up-to-the-minute merchandise. Or she may 
have alternative tasks which affect her sales effectiveness, as, 
for example, to holp keep stock In shape, to assist the buyer, 
and the like. 

One mao has a slightly higher gauge of metal so chat bis 
"poundage" reflects this fact, rather than his "assiduity ” 

Another man's machine, “just like thu one,” rant at a 
■lightly higher rate of spend than hia neighbor’s so that hit 
poundage, and consequently hur pay envelope, reflect thu fact. 

■An ilitmsito to id cmftloy ttz (du cfttnrbi of witch an irbitrarUy quanti- 
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The ‘'units" of production, although assumed equal, may 
not m fact b* equal In punching 80-column Hollerith cards, 
for example, one girl may punch daily 1000 cards and a second 
the same number. If the fint punches full cards (80 columns) 
while the second punches only 40 columns, the “column” mea- 
sure of production reveals the first worker to be approximately 
twice “as good” as the second. 

Even with the units (columns, in the above example) com- 
parable the difficulty (“risk”) of the tasks may be unequal so 
that SOjOOO columns, or aggregate holes punched per day, of the 
one may not be equivalent to 80,000 columns of another. The 
one may be punching data requiring coding, which slows the 
task, or may be punching from a rather illegible data medium, 
while the other may have straight numerical punching from 
highly legible and exceedingly conveniently arranged data 
media, And even if both are numerical the one may hive a 
data-mcdium where the entries are hard to locate (requiring 
many backward and .forward movements of the eyes) while the 
second has a data- medium m which all the answers align m a 
column at the n^it so that punching them it all but an auto- 
matic job. 

Tune tends to relegate many factors such as those men- 
tioned to the status of chance, or compensating errors. Accord- 
ingly, tie longer the time over pluck criteria are collected th* 
less important, generally but not neceasarily, are such "errors " 

In some cates, also, one may to arrange it that there ore ade- 
quate controls of such matters, that, for example, half of the 
subjects work for one week on (he one project while tho other 
half work on the other, and the following week the groups inter- 
change placet, while tome multiple of two weeks' wages is taken 
at the criterion variable. 

It would appear, (hen, that production is likely to be a 
better criterion, if the attendant circumstances of na accumu- 
lation are under the control and supervision of the research 
* worker; if, m a word, he collects the data himself, rather than 
taking uncritically what may be handed tq him by a production 1 
officer or records dork. Only a very few firm* now collect 
routinely multiple, and reasonably comparable, aspects of the 
■uccew of their worken on various variables. 
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3 Quality of Work The quality of work done by a 
"worker” is everywhere highly regarded as an important ab- 
stract virtue. In some cases, pucb as the performance of the 
stage, screen, opera, studio, prize-fight ring, and operating 
table, it is the paramount consideration Yet, like the fanner 
variables, it coo has ita difficulties 

In bonus scheme* bad quality of work frequently u dis- 
counted, and oftentimes more heavily perhaps than the "psy- 
chological" meats of the case would warrant. Such penalties 
often are "set high” as a means of discouraging their occurrence. 
Sometimes they are baaed on a quasi-rational principle as "the 
estimated time, often generous, which it would take to undo the 
incorrect or erroneous work and da it over again correctly or 
'errorlossly.* " 

Quality in most cases is based essentially on subjective 
human judgment. Limit gages sometimes give a highly objec- 
tive status to quality measurements where the precise size of a 
product is an important desideratum in the finished product 
repetitiously produced Evan here the roan whose avenge 
error is .0002 usually receives no more wage, honor, merit, con- 
sideration, or even notice than he whose average error is 0005 
when J0Q1 u the allowable maximum deviation of the product 
from the ideal or perfect measurement One remedy for this is 
to devise measuring machines which reveal instantaneously, 
when a finished product is brought between the jaws of a mea- 
suring device and a constant pleasure automatically is applied 
by means of a faction wheel, the amount of error an each piece, 
and its sign. This yields a distribution of the individual’s 
actual errors the mean of which may be taken as his “inaccu- 
racy,” Such a distribution customarily u not rtwtww^ set-, 
bated by industry It consequently ran be had only by special 
nquiry — involving the specially designed machinery^— intro- 
duced for that purpose 

An objective “rating Kale” may be constructed to measure 
quality. Thus by scaling cookies as to taste, and comparing the < 
taste of each experimentally produced cookie with the taste of 
scaled samples one may ascribe numerical magnitudes to the 
taste of samples of cookies, and so assign tho merit of the several ; 
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fnoki thereof, even though the scale be consumed in the 
.rtrtwl Sewing, soldering, cookie-cutter, wire-splicing and 
EJeim* scales have been used and, when properly used,* yield 
objectivity *n d probably also corresponding validity, in their 
Mjpective realms. r « « 

One may sample the products at periodic intervals, by 
--tint for example, the errors made in successive intervals, 
collected at any time in hourly intervals by the inspector on hia 
wads in successive, frequent, but unanticipated intervals 
throughout the work day. This haa been called the timed 
sampling technique Errors in this may arise if the worker 
antes, consciously or otherwise, to do a better quality of work 
[or a limited time when he auspecta an inspection is “about 
due," 

The best measures of quality result from those situations, 
ia repetitive production, where all errors above an allowable 
minimum are automatically detected by objective inspections 
(eg-, by the electric eye). In this case the poundage of dis- 
carded work is a significant measure of the quality of work 
done. 

i 1( i Breakage and spoilage of raw materials or of partly finished 
poductB to which the worker is fitting additional parts are 
nportaot aspects of a man’s worth and so often are elements 
d a criterion. In assembly or bench work often these consti- 
tute the core of the criterion To take account of different 
ipeeds of assembly of different workers one may take as his 
index of breakage or spoilage the percentage (or proportion) 
which the wasted material is of the total material processed. 

Clerks who otherwise are idea] employees may fail miserably 
m tfieir employer's eyes if they are highly susceptible to making 
many clerical errors, illegibilities of writing, or errors in compu- 
tation, particularly the undetected ones which cost the em- 
ployer money or require him to adjust matters with irate 
customers. 

4, The Rate (or, alternatively, the amount) oj Acquisition 
of New Skills. In aptitude tests, generally, the length of time 

i It fa dun ondooked ihar iT [t takca t* hidgti u deicmuac an object rr* tala 
it or alio take iliamt u many judpa rtlnWy to iKrtba ran* to IndivMhial prod- 
•m by fca aid. 
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taken by persons, equally unskilled at the beginning, to achieve 
equal competence » a rather valid cn tenon measure. This pre- 
supposes “equal opportunity to learn ” It stresses the over- 
head aspects of employee worth In many jobs the training is 
costly. In war-time, for example, delays in production — due to 
training tune, or any other cause — are scanned more closely 
than usual Other things equal, that employee who costs least 
to obtain, to tram and to maintain, is the most valuable. And 
if promotion » contemplated, the fast learners are those most 
worth retaining. There ia at least a fur presumption chat they 
will be rapid learners on the subsequent job employmg a good 
core of the same kinds of skills. 

This criterion may bo applied either to apprentices in a for- 
mal training school or to worker* in training on the job itself. 
In fact, is will be shown below, it u sometimes possible to io 
arrange it that the job itself becomes a prognostic teat with 
generally good all-round rexulca. 

5. Supervisor's Judgments, These may be of "over-all pro- 
ficiency" and may be employed in lieu of a more satisfactory 
objective criterion; or they may be ratings, soy, of the several 
specialized ends or objectives of the work done, of which it is 
generally claimed, or claimed for the present, in lieu of objec- 
tive scales thereof, that only the foreman or other supervisory 
officer is an adequate judge. 

These often are notoriously unreliable in the technical, as 
vs, the popular, sense The judgment of a foreman in an eyelet 
department of a certain brass factory agreed with those of his 
assistant as to the over-all merit of hn men to the extent of only 
60, even though aided by a formal card-selectian-into-pilcs 
method of rating. He agreed only slightly better with himself 
on a subsequent occasion; that is to say the self-correlation was 
but little better. The antidote for such, of course, is repeated 
ratings, monthly or bi-weekly or even weekly ratings over a 
considerable period of time, with all the resultant ratings aggre- 
gated into a composite, automated or averaged, score By this 
expedient the composite rating may be made to approach as 
near 1 00 in reliabihty as we wish while its validity may be 
expeeted to obey the Brown-Spearman prophecy formula; that 
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is to aay, may be expected also to increase somewhat but to 
approach rapidly, with increase in the number of ratings, a fixed 
ceiling which generally is far short of 1.00 The crucial ques- 
tions here are: (a) just what “traits" Shall be so rated, and 
under what circumstances? and (b) how many racings should 
be secured, and from whom? 

The rating of the products of artists' work, pictures, etch- 
ings, and the like, are notoriously unreliable, mtercorrelating 
no higher, perhaps, than JO. Even the reliability of the mueh- 
vaucited medical skill and judgment in diagnosis r» probably 
no higher than 50. In the latter case Brown's formula tells us 
that to secure a composite judgment with a reliability of .95 — 
such as any group intelligence test, in its field, will yield on a 
2-hour examination or thereabouts— not to mention validity, 
would require the independent judgments of an assemblage of 
some nineteen equally competent doctors It follows that (a) 
often one cannot assemble enough “judges" who know well 
enough the subjects of the investigation to rate them accu- 
rately, and (b) that one often cannot, in many fields, obtain 
judgments or ratings “worth their salt." 

The reports of a training supervisor may bo of more worth 
than of a regular line officer Thus the worth of an assistant 
foreman, for purposes of rating job-worth of employees, may be 
greater chan that of hia superior, the fore man- in-charge, par- 
ticularly if the former supervises the men in their day-to-day 
work m the shop, while the latter concerns himself largely, or 
at least more than the former, with the general planning of the 
work and the paperwork of the department. In fact the latter, 
under thu circumstance, may be all too much influenced m his 
ratings by his knowledge Of the wages which they receive and 
, all top little by their more pertinent behaviors under varying 
Shop conditions The size of the pay envelopes or the exalted- 
tress of the titles of the supervisors docs not necessarily correlate 
highly, or even positively, with their ‘‘ability to judge." The 
correlations could be negativel 

If the foreman believes that “experience makes the man," 
ha ratings or rankings will be substantially a ranking of the 
men by age, or by experience And age, or experience, In thia 


cue will correlate more highly with hia Tatrnga than any other 
predictive variable Since age, or experience, in rfoat occupa- 
tions does not correlate highly with “competence” it follows 
that this attitude leade to spurious or attenuated ratings the 
badness of which will not be readily apparent 

It if notoriously true that supervisors of teachers are not 
able to rate reliably— not to mention validly — the teachers 
under their direction on even a fairly objective trait, 1 such as 
the teacher's “ability to gam and hold the attention of the 
class " Consequently we must believe that any generalized 
objective such as the "teachers’ teaching effectiveness" cannot 
be reliably sated, even with the aid of formal rating scales, with- 
out many repeated visits for observation of the actual teaching. 
The ratings actually received often are a composite of the 
rumors about a teacher seasoned only with a dash of real knowl- 
edge thereof. 

The situation in school or industry in which more than 
two — or even two — person* know intimately all those under 
their direction is rare indeed Two ratings by different persons 
are probably more valid than the same number, two, of re- 
peated ratings by one judge, but if the former cannot be ob- 
tained the latter is about the only alternative * 

One thought here, however, where the number of judges 
oF necessity is inadequate, a to have fdiaw- workers, fellow- 
apprentices, fellow-teachers, or fellow-pupils, say, rate one an- 
other. The extra number of nten may either partly or fully 
compensate for any assumed “lack of experience in (or compe- 
tence of) judgment." Such “judges" indeed are able, for exam- 
ple, to note certain aspects of competence not so readily de- 
tected by the supervisory officer. Workmen often kn jw, for 
example, when the shoddy work of a fellow-worker is being 
“buried" by an overdose of a beneficently-hiding coat of paint, 
like the doctor’s mistakes by the casketl 

One might even weight more heavily the judgments ren- 
dered by the adjudged better worker* than those of the poorer; 1 
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but If “everybody judges everybody"— -under conditions Where 
nearly everybody luuws everybody weU-tt « a fair guess that 
the additional ratunw from such weighting generally may not 
pay for the additional time and trouble necessaiy 

6, Knowledge Achievement tests and trade tests of knowl- 
edge possessed may be employed as a criterion sub-variable 
There it mom truth than fiction in the sanding joke that the 
eminent surgeon's bill or *200 for removing your appendix 
rnlly u *2S for “removing your appendix" and *175 for bis 
“knowMge of what to do and what not to do** Popular 
opinion has it that there n » aknoat aero correlation between 
knowledge and skill Actually m moat trades the two correlate 
quite highly} so highly in fact that qcil trade tests (measuring 
knowledge primarily) art excellent tests for picking men in 
order of their ability 4f tc perform" on the job. 

, 7, J 0 b Tenure. This is comparable if the employees, say, 
oil entered the firm at tho same tuna and so have a good chance 
of being subjected to the same (historical) factors and condi- 
tion. This variable a only slightly related to "competenct," 
i Improbably u highly related to that aspect of job success known 
, aa *Uob ratiafncuon.*’ It ’would not be comparable if business 
cycles, ware, or other "disturbances" did not "cover" all env- 
/ ployees’ records equally. . , . , 

Following general reasoning that that profile »mo»t sable 
(statistically reliable)* and moat representative of the person, 
othor things equal, which is most unique (In the factor sense) 
it would sc«m that it could he concluded that one ihould aim 
in predicting profiles to include as many umquo variables os 
possible, while rf one is combining thwn into specific tub-cn- 
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eerion Tariablei, such ii“»raur»cy," then the.aniblamdtij 
*ouM he u “loedid’' with the eub-ciitenon "h* tet“ (hen 
"■ecurecy") u poraible. In the eetoed cae, the eeeune. 
"teeti" ihould be hi, My intercorreleted ntherthen the am. 
veree. 

8 Supervisory and Leadership Ability The normal prog- 
ress of a man m industry— and particularly bo in war tunes— 
is to be given supervisory responsibility as fast as be demon, 
atrates an ability to assume it, Where this is the case the 
Xujterwory status of a man is an important criterion element 
The foreman is distinguishable from the workman often, 0r 
even usually, not so much by "superior ability to do" as' by 
superior knowledge, better Judgment, greater ability to solve 
difficult practical and theoretical problems, ability to* lay out 
work, to aue-up repair jobs, to criticize faulty work, and to 
induce men cheerfully and uncomplainingly to work indus- 
triously 

The number of workers supervised la probably 0 | far less 
moment chan the adequacy, or quality, of the leadership ren- 
dered This conceivably might be measured by toms form of 
check lut. ' 

9. Incidental Factors. 

(a) Amount of supervision required. 

(b) Attitude toward supervision 

(c) Job satisfaction, 

(d j Adjustment to the work and to fellow-workers. 

(a) "Influence* on fellow-workers (e.g., morale-building 
influence). 

(f ) Skills inventory, for example a measure of the "cor- 
rectness of motions," the extent of possession of such a set of 
motions that present efficiency ia likely not Only to persist but 
also to increase (improve with practice ) . Does this auto driver 
have the right "driving habits" as measured by a check lull 
Does this salesgirl have the right sales tactics, as ascertained by 
a professional shopper? Does this profesiional swimmer hare 
the "logically correct swimming movements*? 

Vitelea points out that success is not speed plus accuracy 
plui job tenure, but that it w an integrated whole. The diffi- 
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culty of explaining an s^nmrawnal whole in quantitative 
terms requires as the only alternative to combining the teveral 
scores, as by weighting, far example, that one treat di* sueeeas 
aa a profile' 1 as above indicated- 

Bingham and Freyd have painted out that objective criteria 
are more likely to correlate with objective tests, while rating/ 
are more likely to correlate highly with te*a of personality. 
If SO. ratings, often hitherto employed as criteria, »w doubt 
have perpetuated some testa which ought to have died a natu- 
ral death long ago. 

K criterion that U objective and uninfluenced by human 
judgment is usually "a more predictable measure" of job pro- 
ficiency than a rating based entirely upon rupervUon’ opinions 
of the workers 1 perforouncc. 

A few general rule* applying more or less to all such cri- 
terion elements may not be out of place* 

1 It is important to tnake provision for xhe orderly and syste- 
matic wllecikll of the criterion first, that is, ahead of adminis- 
tering tie tests; otherwise the subjects may “gflt away" from 
on a (by the route of graduation or drop-outs of students, the 
transfer of workers, the shipment overseas of soldiers, and the 
like), whereupon one will find that he hss a fine bunch of ex- 
pensively collected and scored tests papers, but no criterion to 
tell him whether they are worrit anything or not. 

The test scare* cannot of themselves tell one whether they 
an) bj any worth. One of course must have a wide dispersion 
qf test scores, but that of com™ |s a fairly common character- 
istic of worthless tests, The test be fairly difficult, ideally 

of such a difficulty that the average score h about half the pos- 
sible maximum scow, but tjhat also is a characteristic of many 
worthless testa It must have »m,e reliability, but, in the ex- 
perimental stages and other things equal, the reliability had 
better bn low rather than high to the end thst its validity as 
1 w«ll *M its reliability may be greatly improved by lengthening 
it, Among exiting tests generally — and possibly even among 
' tests pt a particular type »e w«l^-there U no correlation of note 
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between reliability and validity Accordingly high reliability 
io itself warrants nothing as to the value of a test. And finally, 
even if a ten has a large dispersion, the right difficulty and • 
good to high reliability— all three— oven that pattern of suppo- 
ritioni merits nothing ss to the value of a test. Validity mar 
be established only by a validity coefficient, in turn determina- 
ble only by the aid of a criterion. 

Much tune and care ihould bo expended on the collection 
of adequate variables, Tho final test scale or prognostic battery 
probably will be no mors valid than the yardstick by which it 
was constructed. And at beat it can be no more valid than the 
square root of Its reliability coefficient. Thu concilium Is 
achieved by manipulation of the formula for the attenuation 
coefficient. 

Possibly as much tune should be spent in devising the cn- 
tenon as in constructing and perfecting the teats. This impor- 
tant part of a research seldom receives half the time or attention 
!e requires or deserves If the criterion is slighted the turn 
spent on the testa is, by so much, largely wasted. 

All criterion variables should be collected over compilable 
times The measurement of improvement, for example, in gen- 
eral should stare at comparable periods in the several persons' 
Individual learning or growth curves 

Criterion variables which accrue in time should represent 
the same amount of rime sampling. It is better, Tor example, 
to Include jn a study all freshmen who persist to the end of the 
year, taking as a criterion score for each his end-of-tho-fresh- 
men year scholarship (to one definite point m dme), than ® 
include also therein tho averaged final euccess of those who 
dropped out with only one quartet' (one-third of a year) <4 
college work and alto the final success of those additional oat* 
who dropped out with only two quarters (two-thirds of ay ***). j 
of college work and so on The three populations mention™, 
have Over-all marks which have vastly different reliability x 
coefficients, sxy 70, ,824, and 875 (Brown’s reliability fonpufs^ J 
respectively, and considerably different validity coeffirifritA 
This is party in the interest of rendering comparable th« * 
ports, for cum pie, the validity coefficients, regrestkw 
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UK,, and the like. If one employe the maximally heteregene- 
HU group, the validity found by another who uses the same 
tun, otherwise comparable, would be exactly comparable only 
if he had th* same fnportumi of the three sub-populptions 

ff * I [f C djta 3 are missing in only a small portion of the cases and 
mo ply a few of the -criterion' variables these may be and per* 
hipi ordeienly fl hould be supplied before beginning to eombuio 
the data Toopi" has developed a formula for this The for- 
omli usuries that for any missing score in a given variable one 
dunld supply the weighted average standard score of the per- 
iod jn question on those variables in which the data are avail- 
ablo, If the variables are highly mtercorrelated, and few scores 
ut musing, the assumption is justified. If a small per cent, 
preferably not over one per cent, of the persons cannot be rated, 
or the ratings (rating slips, for example) are lost, and there is 
pawn to believe that these are a random selection of the total, 
then the data-present coses may all be reduced to cencde ranks 
on the bans of the available Aft, while the miBsuig-data cases 
may be assigned the rank SO. 11 To facilitate such “trantmu- 
unon" on a large scale one may employ Hull's Tables. 11 

One may m some cases devise a mechanical device co count 
the success of workers automatically, possibly even un known st 
to them. Thus a Vender Counter may be so attached to a type- 
wnier or a Hollerith punch that it counts every key-stroke 
produced- If the one machine is employed daily and continu- 
ously by one operator, and one operator only, tiro total strokes 
of anyone day, week, or month, as the case may be, in equal to 
the close-of-busineis reading leas the reading-at-the-beginning- 
of-the-time-interval Over long periods of time oven errors, 
inch as the use of the machine occasionally by an axecutivo 
after hours to type a brief letter and the like, tend to iron out 
K such machines arc taken over by substitute# when the em- 
ployee m question is ill or absent this may introduce appreciable 

“Tooth, Herbert A- "Tbe Bdcctsn at Qnduitt Aiiaann," Pmaimt! 
yf {1921), 47D-471, 

“twenafly half by cfainct ihodd be tunned tbs rink SO and baV &• 

Clerk, AuumJ* Tuting, Appendix 1 Yea ken WadA Book Ctu, 
IBS. Pp. tfl-W2 
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errors into the resulting scores, but less than would bo produced 
by absence and non-use of the machine by anyone if the days 
not worked are not allowed for 

Most systems of allowances for erroro are not particularly 
psychological By means of addends the patterns of errors (or 
of correct responses) may be recorded* 4 and later may be «ub- 
jected to alternative modes of analysis. To all intents and 
purposes the addend code number ir the original performance. 

Rate-set cere and time- setters have many ridiculous customs 
and "allowances" as well as wise and wiser ones Their "stand- 
ard time," may be the shortest of rough gu«w* rather than the 
result of a painstaking analysis They often are overly influ- 
enced by a "star (99 percentile) performance” rather than a 
statistically determined avenge performance. 

In criterion-building we often imply the equality of numer- 
ous “things" which in fact never are equal’ 

1 That the motivation of the subjects Is equal. Thu is a 
particularly important consideration, 

2. That the "nik" is equal We here are confronted with 
such questions as. Do people ride cabe equally frequently In 
thu cab driver’s "district" as In others? The durance traveled 
or the fares collected may nor be an adequate measure if that 
u not the case If the houses are farther apart (as in the newly- 
built-up eecneni of town) the potential nden per mQe are 
fewer but the actual ride™ per mile may be more, 

Ie work available at all times so that thu machine tender's 
wngei are free from error by reason of his pay envelope not 
reflecting, or reflecting inadequately, "lost time"? 

3 That the experience, or practice, or education , » equal 

4 That the human factor, as vs. machine factors, is the one 
which produces the basic variation in entenou scores of the 
various individuals of the criterion group 

5. That the persons employed for a entenou may allowably 
be combined in a society, rather than properly only be divided 
into two or more. We have mentioned sex above. The same 
can be paid for age, and in the case of machine- tendon possibly 
the age of their machines. 
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6. That the work environment u equal Ventilation, noise, 
lighting and other conditions all have differential effects on 
individual workers which in some instances amount to biased 
qrora, or worn. Racial or religious antagonisms sometimes 
divide any industrial department into "good" and “poor" pro- 
ducers 

7 That the progress of work is not impeded by hindering 
factors, such as unavailability of raw materials, transportation, 
power or supplies over which the employee has no control 

Comparable Scores 

Before the several portions of a criterion to be combined are 
Weighted, each person's X-score in each variable must be re- 
duced to comparable scores. For this purpose comparable 
■cores may be defined as any scores that have equal variability, 
Since ''success" at best is au arbitrary variable, one need not 
involve In the definition the requirement of equal moans in the 
several variables, If this » the definition adopted, then rank- 
ings (based In common on all N persons in each different cri- 
terion variable, with "tied scores" outlawed) are comparable 

■cores; and so are groan standard scores or Z*e + and 

also A.D. units, median deviation units, and the like. 

Because of their highly desirable algebraic properties, the 
ability to predict exactly standard enure of estimate, for exam- 
ple, if they are used, practice restricts comparable scores to 
standard scores or some variations thereof, such as Z't, or 
P-Korei, k a^id the like, all of which m common have equal 

standard deviations. Standard score*, ai ■ — , are most fre- 
fl| 

quontly employed and have a property, sometimes valuable, 
that all the criterion variable^ automatically by ]ta use, are 
made to have equal meana as well as equal standard deviations. 

The Arbitrary W**ght*ng Formula 

The arbitrary weighting formula for combining the m sev- 
eral sub- portions of a criterion jnto a criterion 'variable, or the 
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leveral sub-criterion variables into a unitary entenon score, 
accordingly ut 

y .Ml + M* + . . .Ms (1) 

0,0, a_ * 

where X u , X m represent the wt several *ub- portions 

to be sumraated, and Pi, Pa, . . . , P* are arbitrary weights, not 
precluding 'logical" weights, to be accorded the m several por- 
tion i, or variables, respectively 

If one is going to predict a profile instead of a unitary en- 
tenon score, one will naturally desire to secure as adequate an 
"accuracy score" or "speed icare," for example— one of the 
basic variables of the prolife— as possible. It follows that 
possibly one will procure several different accuracy reores for 
each individual, &g., hw September accuracy score, his October 
accuracy score, and his December accuracy score- In this 
event, then, we shall need to combine two or more scores (In 
die above case three) for each “variable’ 1 (such ay speed, accu- 
racy, bonus earnings, etc.) available. Consequently in this 
case we sol] will have the problem of combining scores, end the 
above formula with corresponding adaptation still la appro- 
priate. 

Thus we cannot escape the “weighting" dilemma. Even if 
we have only one entenon score; we spll would weight the 
varisble with Pi - 1, in the bids syitem below, 

J{ re erring Sigiw of Sub-Vanoblts ‘ 

If one is combining “accuracy" scores and “error" scores one 1 
may reveree the signs nf the "error" scores so that they Will 
combine properly with the former This can he done either 
hygivuvg a negative sign to the 0*1 or to the error senna, in the 
bids system below. One of the beat systems u to tranaqiut* 
all the original scores, X/a, to transmuted scores on such * vari- 
able by die formula, 

Xi-tfi-PA, ( (2) 

where 3i Is the bids-tmportance of the trait, a positive mignJ* 
tude, and Kg is taken of such * magnitude that all repotting 
’Xt'-econa are pwftws. It follow* that the rearing ^'Vwfll 1 
all be positive but that the less merfeorfou*, 
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producing penoni) will hive the inullut-ilnd ecora ud via 


versa. 

The Buis System oi Arbitrary Weights jar the 

Pot liens of a Criterion 

The several arbitrary weight* flu P», .... of formula 

(1) moat conveniently can be obtained by the bids system pro- 
pojed by X. L Kelley for weighting factore-of-ment in rating 
the SAT.G to. World War L In ewencc it involves the fol- 
lowing elements: 

a N judges, at least above a minimum of competence, 
ascribe to each of m entenon variables blda-of-relativc-impor- 
tance, In respect to their judged importance, under the con- 
trolling principle that the sum of the bids allotted shill exactly 
equal a certain predetermined quantity, say 100 The 
ation is thus rendered as close as may be to that of, "all other 
things equal, what per cent of the total importance (variance) 
can be arenbed to a deviation of It, in this variable of concern " 
It will be noted that thia is one of the central point* in our 
notions or variance. Consequently it ia to be presumed that 
statisticians who have dealt with this specific problem, after an 
adequate on-the-job study of some weeks or months, may be 
more capable of doing what is statistically demanded than th« 
Tun-of-the-mino foreman or supervisor who is about tha only 
available other source of “expert judges " 

b. AFter the judges have rendered independently thoir Ver- 
dicts of bids on the several traits, preferably m integral num- 
bers adding to 100, one may weight all the judgments of a given 
judge, if desired, according to his assumed cowpetence-at-judg’- 
tng Thui ope could'welght the judgments as the square root, 
or cube root, say, of the judge's ysare of experience m super- 
vision up to some maximum— »ay W yearn. If aevenl expert 
judges are available the simpler procedure is to assume that all 
are equally competent, and accordingly by merely adding the 
stores one in a sense weights them squatty . M Thus in ®vwy 
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case we obtain a weighted (summated) criterion score far each 
individual 

c. A compromise 'set of bids u arrived at, possibly the 
rounded averages of the several judges' bids on a given trait, 
No harm is done, and some gain in interpretation is secured, if 
these are so rounded that their sum also is 100 

d The compromise bids thus secured are the values of fi,, 
Pa, 0i, . . ,0. of formula (1), 

The judges should pay no attention to die riffir of the p’s, 
as m the case of “accuracy” and “errors" above, but should 
judge the traits ae-if-they-were-of-abaolute-eipi. 

Points to be noted ut the ascription of the bids 11 are* 

(1) Variables repeated in other variables (having a high 
correlation with other variables) should receive a low weight. 

(2) Variables which are subject to error, other things being 
equal, should receive lower weights than those not so subject 
to error 

(3) Variables representing an adequate sampling over a 
long period of time should receive a high weight, other things 
being equal, relative to a variable which u a sampling of only 
a short performance of a similar trait. 

(4) Tha bids should be made up independently, ue,, with- 
out the judges conferring during the Original distribution of 
bids 

(5) After each of two or more judges independently has 
distributed 100 bids to the n traits, as much revision, before 
comparing results, job desired should be allowed, 

(6) After the bids are thus secured, a set of compromise 
bids should be made up. These likewise should add up to 100, 
In this construction of the compromise bid, the above first three 
principles, and all other pertinent ones, should be invoked in 
the discussion, to insure that a fair set of bids result. It may 
be found, far instance, that two ideals dash; the use of astu- 
tantships, etc., ai rewards for the encouragement of good 
scholarship, and their use to obtain cheap labor for depart men- 
tal routine duties, far example. It may be found necessary to 

I'Taopi, H*rt»rt A. "Tha Selection of Qraduiu Anlinntt." The Ftrnmmd 
Journal, VlilWi) , 457-472. 
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make some distinction between three two; even to the extent 
of establishing two sets of weights, and evaluating each candi- 
date’s menu from the two points of view, successively It 
would seem to bs good departmental policy, in that event, to 
evaluate for both appointments every candidate applying for 
either of the two types of appointments, irrespective of tho fact 
that the candidate applied for only the one appointment. 
Otherwise, from the department's point of view a capable assist- 
ant, for example, may be lost if there are not enough scholar- 
ships and fellowships "to go round”, while from the student's 
point of view, often he will find the alternative position offered 
him to be very acceptable. 


S 

* PERSONALITY AND INTEREST FACTORS IN 
DENTAL SCHOOL SUCCESS 

CLAUDE EDWARD THOMPSON 


Tee purpose of this study was to determine whether or not 
certain criteria of success in Dental School are significantly 
related to tcorea on personality and interest scales. Previous 
research by the writer (15) on the relationships between motor 
and mechanical abilities and success In Dental School required 
that he interview personally over one hundred practicing den- 
tists sad the faculty of a College of Dentistry. These men 
were almost unanimous in claiming that personality and inter- 
est factors are of as much importance as aptitudes in determin- 
ing success m dentistry. 1 

Published reports of research and reviews of the status of 
selection and counseling techniques in dental schools (1, 2, 4, 
5, 6, 7, 8, 12, 13, 14) indicate also that measures of personality 
and interest would be of value in selecting and counseling. The 
Strong Vocational Interest Blank for Men is being used as a 
predictor item lor success in dentistry at the University of 
Maryland. However, no published reports are yet available 
to reveal relationship* between scores on tho interest scale and 
criteria of success in this school 

, Test Used 

During 1942-43, three tests were administered to students 
m the College of Dentistry at Northwestern University 1 These 
teste were 

(1) Preference Record— Form AS, by G, Frederic Kuder 
(3, 9, 10, 17). 
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( 2 ) California Test of Personalty— Adult Senes, devised 
bj Ernest W Tiegs, Welles W Dark, and Leins P, Thorpe 

(16). w * * 

(3) MacQvame Test for Mechanical Ability, by T W. 

Macpnam’e (II) 

flieii iecQuamt Test for Mechanical Ability and the Cahfornta 
TestofPer.ro/udity were administered to 158 freshmen, The 
MtcQuanie, the California Test of Personality, and the Kuder 
Preference Record were administered to 66 seniors 

Criteria 

Previous studies (7, 8) at the College of Dentistry, North- 
ffutein University, indicated that the MacQvarrie was a usable 
tut for predicting freshman year average grade, Following a 
(Digestion in those studies, an attempt was made in the present 
study to refine criteria for test evaluation by separating theo- 
retical and technique grades From practicum grades This was 
done by obtaining cumulative points earned in technique and 
theory and cumulative points earned on product or work done. 
The scores were arranged in ascending order and divided into 
deales ft was then possible to score anywhere from 1 to 10 in 
both knds of grades. 

Results 

Table 1 presents the correlations obtained for the 15(3 fresh- 

* 

Table i 

ComUtbv of T/it end Criltnon Scorn for 1$> fttfkmt* 


Thoory ud Tacholqn* Pnctmun 


MkOkhttc Tan! Scon . . 
M-AiHuticmt T«»l 


men between theory and technique and practicum criterion 
scores and scores on the two tests The CaUfonua Test of Per- 
sonality gives Self-Adjustment, Social-Adjustment, and Total 
Adjustment scores These were correlated separately. It 


should be pointed out chat the California Personality Test (16) 
has not been validated against objective outside criteria Hie 
1BQ items were evaluated in the following manner* 

(1) Judgments of teachers, principals, test experts, person- 
nel directors, and employers as to whether or not each item was 
an indicator of adjustment and employability 

(2) The reactions of employed adults as to whether or not 
they judged each item to be an essential characteristic of a suc- 
cessful employee. 

(3) The extent to which the results of the test agreed with 
the known characteristics of particular adults. 

(4) The extent to which each item was consistent with the 
score on the test as a whole (bi-scrial r) 

Due to the questionable reliabilities of the six components 
in each of the categories Self-Adjustment and Social- Adjust- 
ment, it wsi decided to use only the over-all scores for pur- 
poses of group comparison. The split-halva reliabilities of 
these totals an; 

Sec. 1 Self-Adjustment .BBS 

Sec 2. Soaal-Adiiutment .893 

Total Adjustment 918 

Only two of the correlations approach statistical signifi- 
cance For the 158 freshmen there is no relationship between 
total score on the MacQuorn* and standings in the criteria 
There is positive but low correlation between Practicum cri- 
terion scores and Social-Adjustment and Total Adjustment 
scores on the California Test of Personality, 

Table 2 presents the correlations between criterion scores 
and teat acores for 66 seniors. Only the three of nine corajm- 
n en ts of the Kuder Preference Record on which seniors averaged 
definitely above the norma for this scale were correlated with 
the criterion scores. 

For both freshmen and seniors (Tables 1 and 2) the Mac- 
Quame total scores have little or no relationship to cither Theo- 
retical and Technique or Practicum criterion scores. It is possi- 
ble that the first three sub-tesrs of the MaeQtutrrle measure 
motor skills End the last four sub-tests measure mechanical 
ability Using total scores might) therefore, obscure relarion- 
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TABLE 1 

Comb Hoar of Till end CHlorWn Scans for 66 8tntcrrt 


Theory tad TechnJqea Pnearan 



■hips of rub-tests to criterion scores Total scores were com- 
puted separately for the Tracing, Dotting, and Tapping tests 
and for the Copying, Location, Blocks, and Pursuit testa 
These sub-total scores were then correlated with the criteria 
These correlations are presented in Tables 3 and 4. 

TABLE 3 

CtfrilOKu of Mttffoonii SMt-foul Stow *niA C riitm tor SSI 


Theory and Tcc frreqno Pnwfcum 

Tndnf, Tippm*, Dottle* ... - 17 

Copyint, Lacadoo, Bloc fa, Poiwlt , , .16 -E3 

It can be aeen that consistently, for both freshmen and 
seniors, Tracing, Tapping, and Dotting scores correlate nega- 
tively with Theoty and Technique and positively with Practi- 
aiiti scores, and Copying, Location, Blocks, and Pursuit corre- 
late positively with Theory and Technique and negatively with 
Practicum scores. Five of these eight correlations approach 
statistical significance. If the fine three sub-tests measure 
motor dexterity and the last four sub-testa measure perceptual 
Of visualizing abilities, Theory and Technique scores could be 
expected to correlate with the last Tour sub-tests and Practicum 
scores could be expected to correlate with the first three qjib- 


TABLE 4 

Ctmlarianr ef Jd»rO*enU 9*b-T*ul Bemr nUk CnirrU for 66 Stmvs 
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tests. This expectation receives support from the positive and 
negative sign directions of the obtained correlations, though 
such breakdown does not provide correlations of sufficient mag- 
nitude for use in individual prediction. All correlations (Table 
2) between scores for seniors in Self-Adjustment, Social-Adjust- 
ment, and Total Adjustment scorn on the California Test of 
Personality and Theory and Technique and Practicum criterion 
scores are positive but low. The correlations between Practi- 
cum criterion scores and Self-Adjustment, Social-Adjustment, 
and Total Adjustment scores approach statistical significance 
The correlations between Self-Adjustment, Social-Adjustment, 
and Total Adjustment scores and Theory and Technique ^ap- 
proach statistics] significance. These findings indicate rather 
TAHLE J 

jIwtu* PtrtrmSlU Sto*Ji*ti of 66 St* Ion (% Compnnti of the 
Ktdtr Prrftmtt Jt ami 


Campooa&ti Avtfip Puceafib ObuJaad 



consistent relationships between what tc measured by the test 
and standings in the criteria. However, these correlation* 
would nor be useful for individual predictions. - 

Table 5 presents the average percentile standing of the 66 
seniors jn the components of the Kuder Preference Record. 

The average senior scores above average in Mechanical (91 
percentile), Scientific (93 percentile), and Social Service (67 
percentile) on the Kuder Preference Record, Mechanical Inter- 
est scores do not correlate with either Theory and Technique 
or Practicum criterion scores (Tabkt 2). Seientifiq interest 
scores' correlate positively and significantly with Theory and 
Technique criterion scores but not with Practicum, and Social 
Service interest scores correlate positively and significantly with 
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Practicum criterion scores hut not with Theory and Technique. 
These findings indicate that interest patterns are related to the 
marks earned in Dental School, hut the relationships appear to 
bo more specific than the relationships between personality 
measures and marks earned 

The failure of the scores in the three components of the 
JCitder Preference Record to correlate more highly with cn- 
tenon scores could have been due to the narrow Bpread of the 
scores (see average percentiles, Table 5), particularly in Me- 
chanical and Scientific interest scores, It is well known that 
the coefficient of correlation is affected by the variability of 
.scores in the group tested To teat this idea adequately it will 
be necessary to administer the Presence Record to a group of 
freshmen and a group of seniors in a follow-up study 

'Hie deviation of an individual’s interest scores from pattern 
(Table 5) may also be an index of the extent to which the scale 
places those freshmen and seniors standing low in Mechanical, 
Scientific, and Social Service interest scores low in criterion 
scores and pliices those scoring high in these same components 
high,in criterion scores! 

To determine whether or not there were statistically signifi- 
cant differences between tho mean scores of freshmen and 
union on the MncQuarne and the California Tm of Person- 
ality, critical ratios were obtained These critical ratios Indi- 
cate that there is no significant group difference between fresh- 
men and seniors on the MocQvamc. There ■ a significant 
difference in favor of the seniors in Self-Adjustment scores on 
the California Test of Personality. There are 94/100 chances 
that a difference in favor of the seniors in Social-Adjustment 
.cores i* real. There are 99AC0 chances that a difference in 
favor of the seiuare In Total Adjustment scores is real 

Sumwiry 

(1) Seniors in dentistry score above average in Mechanical 
(91 percentile), Scientific (93 percentile), and Social Service 
(6? percentile) interest scores an the Kuder Preference Record, 
Mechanical interest scores do not correlate with either Theory 
and Technique or Practicum criterion scores. Scientific uiter- 


F ACTORS IN DENTAL SCHOOL SUCCESS 

est scores correlate positively with Theory and Technique cri- 
terion scores and Social Service interest scores correlate pan. 
tively with practicum criterion scores. 

(2) Statistically significant differences in favor of the den- 
tistry seniors over dentistry freshmen in Self-Adjustment, 
Social-Adjustment, and Total Adjustment scores on the Calf, 
forma Test of Personality were obtamed Whether or not thee 
differences are due to selection, age, training, or all three n not 
known 

(3) Tho California Test of Personality gave statistically 
significant positivo correlations between Practicum enterion 
scores and Social-Adjustment and Total Adjustment scores for 
freshmen and between Practicum criterion scores and ScH- 
Adjustment, Social-Adjustment, and Total Adjustment scores 
for seniors Statistically significant positive correlations be- 
tween Theory and Technique criterion bcotcb and Self-Adjust- 
ment, Social-Adjustment and Total Adjustment scores were 
obtained for seniors, 

(4) The results of this investigation indicate that correlat- 
ing MacQnorrts total scores With cn tena may obscure relation- 
ships of sub-test scores to criteria 

Personality and interest scale scores show some relationship 
in this study to criteria of success in Dental School, but the cor- 
relations ere not of sufficient magnitude to be useful in indi- 
vidual prediction When selecting applicants for admission to 
the College of Dentistry. 
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THE WORD-DEXTERITY TEST, A BETTER 
MEASURE OF COLLEGE 
APTITUDE 

. SHAILE8 PETERSON 

The Unwenuy of Chiaio 

TBa purpose of thU amcle is to describe a measurusg instru- 
ment which, » an aptitude test, compares favorably with ex- 
aminations commonly given dunng Freshman Week. This test 
has been tried at jumor-high-tchool, semor-high-rehool and col- 
lege level and has proved itself a valuable predictor of school 
and cauise grades 

Tho origin of this examination dates horn work in remedial 
reading carried on at the University of Oregon in 1933, in coop- 
eration With the late Dr. DeBiult. Experimentation indicated 
that there wee a distinct improvement in working vocabulary 
as soon as the student could see the tarry-over in word meaning 
from one word to another. While this seemed to ho particu- 
larly marked in the field of tho natural science*, other work ha* 
indicated that this ability aida students in other areas. The 
first mstnimenta were not teats hut instead were primarily 
teaching devices and intended for students who required reme- 
dial assistance to improve their school marks. New instruc- 
tional devices were prepared at Lebanon High School and at 
the University of Oregon High School At this time, the author 
described these first efforts in The English Journal} Later at 
the University of Minnesota, with die encouragement oi Of. 
Alvin C Eunch, Dr- Guy Bond, and Dr. Palmer 0 Johnson, 
more experimental work was conducted on this same project. 
Still more recently at South Dakota State College there wv 
further opportunity for the author to obiervo the value of the 
Word-DanUnty Taft that had been developed. 

* Ptunon, SbaBcr A Tndikw 8p*ekJ VaabahiW TMKedfAJW- 
ul ifSeSne AWmJ.XXV (WMJ.SW*. 
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THE WORD-pBXTBRITV TEST 


Ths main objectives of the teat in iu present form are to 
foemlne to what extent the student knows the meaning of 
nrtun suffixes and prefixes in common use and also to deter- 
mine if bo can transplant the meaning of a suffix or prefix found 
w one word, whose meaning ib known, to another word in which 
^ iamB suffix or prefix is found F rom an examination of the 
Htt items illustrated m this article, the reader can understand 
while pure memory of words, suffixes, and prefixes will 
loiit the student in securing a high score, still when the range 
gfword difficulty Is at the proper level, the test becomes easen- 
uilly a problem for the student to demonstrate hu ‘'dexterity” 
at manipulating word parts and word meanings. Mechanically, 
tho also becomes an “analogy” type test or 6no which tests for 
configuration and pattern, 

From an examination of the following directions and sample 
tat items, the reader will be able to understand the conitruc- 
umt of the entirt test. It is a power test rather than a speed 
tot and the fifty-item test described here could be administered 
to umor-high-school or college students in 40 minutes. The 
jordiuied in the examination were chosen after consideration 
of iheir Thorndike word count These words, in addition to 
hiving a fairly extended range of difficulty, all employed suffixes 
md prefixes which in turn were to be found in at least four or 
five very common word# as revealed by their Thorndike word 
count, v 

Direction/ 

Tn tfni teat, there arc many scientific words that you already 
know Of the others, you will find that in many cases you will be 
lUa to discover their meaning as you proceed, 

In the example, the word part SUB is considered. There are 
■any words in our English language containing SUB Ono of theeo 
Midi, SUBMARINE, has bean printed on the Une alongside. There 
h ■ ipacn for two other words containi ng th is nine word fart, SUB, 
ind in thu example, the two words, SUBWAY and SUBNORMAL 
hn been written In. 



Each of Che three words, SUBMARINE, SUBWAY, and SUB- 
NORMAL) contain the same common part SUB The fint word 


refers to an under-water boat, the second word refers to an «ndsr- 
gronnd railway; while the lut ward refers to things or condition* that 
are btlota or under normal All three of these words not only have 
the same word part, SUB, but they all have the arme meaning, 
"voder 

On another lino bcaide each of the sections, there fi a group of 
deftmdoni, one of which is correct. You are to select the correct 
definition and place ua letter in die apace in the margin. As you will 
see in the following part of the example, definition "fi” best d escribes 
the word part, SUB 


U) 

m(i . _ 

sssa. 


■ttd watte, uMUlnti (B) tnuth, Mis*. 

— “' 1(0 MihtetUca, Mi*tuui (a) Aaw, 

(g) slxllip — -***- . 


While there » a space in each section for you to write only two 
additional words, you may be able to think of more than two, In 
•ome case# you may have difficulty in thinking of more than ono ad- 
dloonsl one The more words that you can think of, the easier it 
WiU he to deads what dm true meaning is, for as in the case of the 
words submarine, subway, and subnormal, each wmd is alike iu two 
respect* There words have tha similar word pan SUB and also 
have the rimllir meaning, “under” 

You will And that this Is true of other words alio. Whether you 
can think of five new words or only one or cwo, try to deads on the 
best definition for the word pan under consideration. 

DONT WASTE TIME. DO FIRST THOSE ITEMS 
WHICH ARE EASIEST FOR YOU. 


3. MIS 

4 00 

S. 1ST 


6. LOGY 


Ten Items 
misunderstand 


(A) wrong*, (B) small, tiny, petty, (C) ex- 
cure, reason; (D) denote# feminine gender; 
(E) poor, inferior 

cooperate 1 

(A) with, together; (B) two, (C) without 
•topping, (D) aid, help; (E) easy, ready 


(A) science, logical study, inveinntion, (B) 
schooling, training, preparation; (C) forming 
m; (D) helping, am sting, aiding; 


agent nouns; 1 
(E) including, 
mineralogy 


(A) weather, climate, temperature. (B) 
Inti (C) prediction, estimation, (D) duct 
theory, doctrine; (E) bred, Weary, alow 
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10 GRAPH biography 

11 ANTI 

32, 8CRTB 

34. POLY 

39 GEN 

44. PATH 

47. DECI 

48. DUCT 

49. ANTE 

In these items, the student is asked to write down a group 
nf other words each of which contains some of the same prefixes 
and suffixes. This 1* to assist him in assigning woid meaning 
to the word parts in the unknown word, The t eet is scored, 
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however, on the basis of correct responses to the word's mean- 
ing and ordinanly no attention ta paid to the particular “assist u 
words that he writes down. If thu examination were adminis- 
tered in a situation where attention was to be given to remedial 
work, then the character of these "assist” words would be 
Important. 

Table 1 shows the product-moment correlation coefficients 
between different vanatea in a college group of nearly three 
TABU 1 

ComUilan CtifisienU between Merest Terietu Is Celine Greet 



hundred itudenta From this table, it is evident (bit the Word 
D&tanty Tost predicts grade-point average, Engjiah grades, 
mathematics grades, or chemistry grades better than most of 
the other examinations with which it was compared. Grade* 
themselves in some subjects were better predktore of total 
grade-point average, but this can be explained by the fact that 
tha individual course grades were themselves a put of the 
grand avenge. 

At die high-school level, the product-moment correlation 
between the Word Dexterity Examination and LQ. was +^S 


(A) living dungs, Ida, (B) important, 
guuhed; (C) soil, ground, rocks; (D) 
fog, writing; (E) a study, a science 


diatm- 

draw- 


(A) treatment, medical aid, (B) old, hiiton- 
cal, not used; (C) in front of, before, preced- 
ing! (D) opposite, against, Instead ol; (E) a 
study, a science, 
transcribe 

(A) radio, phonograph; (B) circular, round, 
(C) write; (D) duplicate, (E) vole*, vocal 

polytechnic _ 

(A) anthmeric, numbers! (B) many, much, 
often; (C) few. not many, (D) figures, dia- 
gram*! (E) college, school, study. 

genealogy ^ 

(A) saenoe, study; (B) birth, boni, descent; 
(C) related, corresponding; (D) electrical; 
(E) Bible, Biblical 
pathology 

(A) scientific, (B) anger; (C) suffering; dia- 
cim, <D) Judgment, cadmarion, (E) germs, 
bacteria. 

deqmal . 

(A) anthmeric procaas, multiplication, etc.; 

(B) 'hilf; (C) tsn, ten ones, (D) almost, 
nearly, quite, (E) accurate, scientific, sound. 

ductile 

(A) save, preserve; (B) itmch, lengthei, 

(C) press. Batten, sauce**; (D) able to be 
attracted, attractive, (E) lead, direct, guide. 


(A) outside, beyond; (B) behind, beside, (C) 
in front of, preceding, before; (D) old, buton- 
cal, not used! (E) not, never, none. 
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find with mental age, it wm + 70. The point bisenal correla- 
tion* with years of sderce wsb + .20 and with yean of foreign 
language was T 39"' The bisenal correlation with science grades 
was + 52. 

While the scores for the 7th- and 8th-grade students are 
considerably lower than those for any of the other groups, it is 
interesting to observe that there is considerable overlapping of 
the scores for the upper grades Figure 1 illustrates the per- 
centile rank curves for the seven grade levels. 



PiKtatUi Xtnk 
Flora* 1 

P attaill* Rink Carra lax Fnmn Wort Downy Tat 


The estimated reliability of the forty-minute Wonl-Dex- 
terity Test was + 90 to +. 93 depending upon the groups These 
computations were made by the Hoyt method* and by Formula 
20 of the Kuder-Richanlson method,* each of which gives 
underestimates of the true reliability, In order to evaluate the 
function of each of the items in the examination, the item-test 


»Hoyr, Cyril *TH« RdmMiiy Eiranittd by Aralyi* d Varanra." Piytko- 

W,r ^lchai*Jr! , M S Roden <5 F "Th* CxIroUrion of Tat RduUity 
Coeffinoat* Bind. oa *® Method^ Riibftil Equsralwe*," /annul of ISkmHomI 
PtycJuJcty, m (1919) , <83-681 


TIM WORD-DEXTERITY TEST 


correlation for each of the items was computed both on the 
examination administered at the high-school level and on the 
one administered at the college level Table 2 provides a fre- 


TABLE 2 

Dtaribithoa of CowlstfciU Bved upon Top o+d Bottom. QvttUrt 


Iiem-Tm Cmidvtna 


+ 8 to 9 
7 to Jl 
Am 7 
A tii 6 
4 to S 
3 m 4 
2 to J 
I to 2 
U tn 1 

Medao Value of r 


Frtqocacy of Ilrjnj 


Hi*h -School Group 


Coll»|i Oiunp 


quency distribution of the fifty items according to the correla- 
tions that they were found to have with the total test, a mea- 
sure of item validity when the total test ia the criterion measure, 
The high item-test correlations for all Items in the examination 
must be interpreted to mean that there is a commonality of 
purpose and function for all items in the examination and that 
whatever is being measured by the enure examination is being 
contributed to materially by each item in the examination. 

Sample copies of the author's W ord-Dsxtcrity Tut are 
available directly from him and permission to duplicate this 
examination can be secured. 
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A STUDY OF THE KUDER PREFERENCE RECORD 

DANIEL J BOLANOWGH ami CHARLES H. GOODMAN 
Radio Cofporanon «d Ara*na 

At the beginning of die war, one of the major problems that 
faced industry was the need for highly trained technical per- 
sonnel, The Radio Corporation of America, like many other 
companies faced with a shortage of engineers, planned to meet 
this need with a abort intensive training program for young 
women The program sponsored by RCA was a ten-month 
coqree m the theory and practice of electronic engineering* and 
was given at Purdue University 

Because of the importance of obtaining a maximum number 
of technical personnel who would he successful m the course, 
psychological testa were used as part of the selection process. 
The findings which were obtained as a result of the use of throe 
psychological rests will be published at a later date. 

Three hundred applicants from various colleges of thirty-six 
states and from RGA’s six manufacturing plants applied for the 
course. Of this number eighty-six were finally selected The 
girls selected were called RCA Cadettes Their ages ranged 
from eighteen td twenty-nine years, with a median age of nine- 
teen years. Five Cadettes had had no previous college experi- 
ence, sixty -five of the group had either one or two yean of cob 
lege work when they entered the course, and one Cadctte had 
had graduate work. 

pie positions which the Cadettes wore expected to fill were 
varied and included such activities as writing instruction man- 
uals, design, drafting, statistical analysis, laboratory assembly, 
and equipment testing Because of die variety of Jobs open to 
the Cadettes It was felt that it wquld be helpful m placing them 



SIB 


if their interests could be determined. One month after the 
Cadettes had been at Purdue University the Kuder Preference 
Record was administered to them The reasons for using the 
Kuder were' (I) it offered interest areas which appeared 
related to the jobs that were to be filled; (2) it offered possi- 
bilities of determining the preferences of the Cadettes for plac- 
ment, (3 ) it offered an opportunity to examine the Kuder as a 



Fkl 1, Median raw kmw of CnJettra cwrorted ion pmanfe «aW«9 

as lh»Kuin norm*. 

possible selection device for future training programs, and (4) 
it was easy to score. 

The results reported in this paper are based upon the scores 
obtained from the sixty-eix Cadettes for whom there were com- 
plete data Upon completion of the scoring of (he Preference 
Recordj an mdlviduol profile was constructed for each Cadetre. 
In order to obtain a graphic profile that would be represent!- 
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^ of the Cadettes as a group the median acorea were calcu- 
lited for the Cadettes on each category of the Preference 
Rewrd. 

Figure 1 shows the profile constructed by plotting these 
median scores after converting them into the percentile equiva- 
lents given by Kuder in his norms for college freshmen 1 It is 
Interesting to note from Figure 1 that the Cadettca as a group 
itand at the 84th percentile in the Mechanical preference area 
and at the 92nd percentile in the Scientific preference area 
There is, of course, the possibility that the Cadettea' responses 
may have been influenced by their desire to appear highly mter- 
mted in scientific and mechanical pursuits because of the nature 
of the course they were taking. It should be remembered, 
however, that the Kuder Preference Record played no part in 
die selection process of the Cadettes. The Cadettee appear to 
be similar to the norm population in Computational, Artistic, 
ind Social Service preferences. They are, however, consider- 
ably below the median of the norm population in Persuasive, 
Clerical, and Musical preferences. 

The graphs in Figure 2 show the percentage distributions 
of the Cadettes* scores on each of the preference categories 
The norm graph shown was constructed m order to facilitate a 
visual comparison of the percentage distributions found for the 
Cadettes on each of the preference categories. The base lines 
nf each graph have been divided at interval points correspond- 
ing to the 10th, 25th, 50th, 75th, and 90th percentiles for the 
norm group Each graph can then be compared to the norm 
graph. 

Figure 2 appears to support the evidence of Figure 1 of the 
selection of a group particularly interested in the Scientific and 
Mechanical categories, a finding similar to that obtained by 
Goodman* for engineering students. There are, however, a few 
Cadettes helow the norm medians on these keys The distri- 
bution of the percentages on the Clerical graph is the reverse 
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of those on the Mechanical and Scientific categories Forty- 
four per qmc of the Cadettes fall below the 10th percentile and 
only four per cent are above the 75th percentile. The majority 
of the Cadettes Fall below the 25 th percentile in Persuasive 
preference, although there are five per cent above the 90th per- 
centile who might fit into jobs where considerable contact with 
others is necessary, such as quality control inspectors. The 
Cadettes tend to fall in the middle ranges in Computational 
Preferences 

These graphs were useful in helping to determine fippt'place- 
raent of the Cadettes. The Cadettes in the extrema ten per 
cent groups of each Preference category were given particular 
attention for possible placement in jobs related to their prefer- 
ences. For example, some of tho jobs io be filled involved 
mechanical work, scientific work, clerical work and literary 
work The histograms in Figure 2 wore also used in counseling 
the Cadettes and aiding them in the interpretation of their own 
Individual profiles 

To evaluate the possibilities of the Preference Record as a 
selection cool for similar programs, correlations of the Cadettes’ 
Preference scores with their final grade averages for the entire 
course were obtained. It was found that first- and second- 
acmes ror grade averages of the Cadettea correlated .81, which 
would indicate fairly close agreement between tho grades of the 
first and second semesters. Aa a result total grado averages 
were taken as the criterion. Table 1 gives the frequency dis- 
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tribution of grade averages, and shows a range from 3 0 to 5.9. 
The grades for each subject were given in whole numbers rang- 
ing from 2 to 6. Grade avenges were obtained by multiplying 
each subject grade by the number of credit hours and dividing 
the total for all subjects by the total number of hour* Tor die 
course 

Table 2 shows the correlations of grade averages with the 
various Preference scores. Since none of the corrdationi attain 
statistical significance* (even at the five per cent Jove)) the 
Preference scores cannot be considered as predictive of success 
m the Cadetto training course. 

For further clarification of the relationship between Prefer- 
ence scores and success in the training course, an analysis was 

TABLE 2 
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made of the scores of three sub-groups— (1) the most success- 
ful students, (2) the least successful students, And (3) those 
who terminated thelrjraming before the course was completed 
Figure 3 graphically shofn the median scores converted into 
their percentile equivalents on the Kuder norms of 16 Cadettqs 
with grade averages of 4,9 or above, 13 Cadettes With grade 
averages of 3.8 and below, and the 8 Cadettes who dropped out 
of the course for reasons other than those attributed to illness. 
For groups as small as these, the proximity of the curves for the 
most successful and least successful Cadettes who completed 
tho couree appears pronounced. Figure 3 seems to bear out the 
low correlations of Table 2 The la (gen differences between 

„ * LMmirt. E F $i *UrfM Adkl,* fa MAeothed MutmL Now Yurtr 
Houston MWd Coh IBM. P 212. 
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the most successful and the least successful Cadettes appear to 
he on the Computational, Artistic, Musical, and Social Service 
categories 

The profile of the terminated Cadettes appean to deviate 
from the patterns of the groups who completed the course. The 
terminated Cadettea have lowei median percentile scores in 



Social Service categories. 

In order to determine the significance of the differences 
shown in Figure 3, t ratios were computed using Student's tech- 
nique for small sample distributions Table 3 shows the median 
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raw scores Tor the three sub-groups on each of the Kudtr prefer, 
enco scales The t ratios obtained for each group, when com. 
pared with the other two groups and the probability values for 
these t ratios, are shown in Table 4 

Accepting Fishers criterion of a 5% probability at indica- 
tive of a significant difference, n will be found that there are 


TABLE 3 
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only fire such P values In Table 4 The five instances of ng. 
nificant differences are 

Computational-— Most successful Cadettea score signlfir 
cantly higher than the least successful 
Cadettea. 

Computational — Most successful Cadettea scorn signifi- 
cantly higher than the terminated Ca- 
dettea. 

Scientifio— Most successful Cadcttes score signifi- 

cantly higher than terminated Cadettea. 

Persuasive— 'Terminated Cadettea score significantly 
higher than most successful Cadettea, 

Persuasive — Terminated Cadettea score significantly 

higher than least successful Cadette*. 
The results indicate that the most successful Cadettea are dif- 
ferentiated from the laait successful Cadette* on the Computa- 
tional Jtsy only The moic successful Cadettea are differenti- 
ated from the terminated Cadettea on the Computational and 


kodes raraiHNCB record 


323 


324 EDUCATIONAL AMD tBYCBOLOQlCAL MEASUREMENT 


s *■ 

S si 

* 


> 5 

1 1 


II 

S R 

* 



s 

1 , 

A. 

e S 

9, 


« * 

i 

i 

11 *■ 

* * 

* 

II 

« 3 

» i 

9 

US *- 

* Si 

s 

if r| A 

9 21 

i 

q 

ill 

5 SI 

* 


8 3 

1 l 

8 

l 

iil ' 

* s 

* 


5 5 

R - 

i 1 ! 3 

s' * 

q 


3 a 

' s 


* * 

s 


- a a 


H ii 

i ii 

ill 

i «j 




Scientific keys The terminated group Is differentiated from 
the most successful and least successful Cadettea by higher 
scorn on the Persuasive scale. On the whole the group of 
terminated Cadettea appears to approximate more neatly the 
pattern of preferences of Kudcr*i college freshmen 

The above differences provide the only evidence that may 
indicate any possible utility of the Kuder Preference Record aa 
a selection device for future applicants in a training course such 
as the one described, 

Summary 

This study has presented data based upon the Kuder Prefer- 
•net Record scores of 66 Cadettea enrolled in a training pro- 
gram for electrical engineering aides. Investigation was made 
of the possibilities of the Kuder Preference Record as a selection 
device and us usefulness as a counseling and placement tool 
The findings show that the Cadettea, as a group, differed' from 
the college freshmen norm group, having particularly strong 
preferences in Mechinical end Scientific purauits There was 
a pronounced lack of preferences in Persuasive, Clerical, and 
Musical endeavors. Score distributions showed, however, wide 
differences among the Cadettes on each scale. There were, 
however, a few Cadettea who showed a lack of preferences m 
Scientific and Mechanical fields while a few ranked very high' 
in Clerical, Computational, Literary, and Persuasive prefer- 
ences. These distributions, together with the individual pro- 
files, provided some indications Tor counseling and placing the 
Cadettes. 

The acales of the Preference Record showed low correlations > 
with final grade averages for the course. The highest correla- 
tion was .18 for the Computational key with final grade svep.' 
age. ; 

The profiles of the most successful Cadettea and the least 
successful wit similar. The group profile of the terminated 
Cadettes approached that of the average college freshmfijJ 1 ' 
These terminated Cadettes made significantly lower acojeq on - 
the Scientific and Computations! preferences than the most 
successful Cadettes. The terminated Cadettea made sigoJfh 1 
cantly higher aeons than die moat successful and least suk***? ^ 
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jjj Cadettes on the Persuasive scale. The moat successful 
Odettes made significantly higher scores than the least suc- 
glffu) Csdettes on the Computational key 

The following conclusions appear to be justified 1 

( 1 ) On the bans of the correlations with total grade ayer- 
qd, the Kuder Preference Record does not appear to be a 
Misniring selection device for predicting course nchiavement of 
{disk engineering Cadette* It does, however, show some 
munise as a device far eliminating those who would be likely 
todiep out before completion of the course 

(2) The Kuder scores afford Some indications that can be 
helpful in counseling and placement, especially in a situation 
nfitre there is a variety of joh openings 
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Our national survey showed that service rating has proved 
of real value in raising civil service standards It is desired 
generally by competent employees since recognition may be 
given to faithfulness and merit as factors in promotional exam- 
inations as well as in eligibility for salary increases, transfers, 
leaves of absence, and other civil service privileges. 

From the appointing officer's viewpoint, service rating pro- 
vides an impartial yardstick by which he can measure employee 
performance, and also provides a definite incentive by which 
the majority of employees seek improvement, to that extent 
bettering the performance of hu department and at the same 
□me simplifying his personnel problem. From this definite 
national consensus, we felt assured in following through with 
development and inauguration of out own system 

We also found that regardless of the form of the rating 
device there are, In the opinion of administrators generally, 
certain requisites to the success of any employee rating plan* 

Ftriv The supervisors or executives who are asked to rate 
dteir subordinates must be sold on the idea- This result may 
be brought about by a series of conferences with such super- 
visors conducted by officials of the personnel department. 

Second The employees themselves should be sold on the 
idea. When merit racings are properly sold^invettigatore re- 
port that a large percentage of employed like to be rated and 
will give complete co-operation to any rating plan that they 
consider to be fair. 

Third' The supervisor* must be’ trained to rate thaw under 
them. One of the principal weaknesses of rating systems ap- 
pear! to be the lack qf uniformity 6f ratings between different 
departments or bureaus In one department, for example, a 
supervisor may set very low standards of performance and rate 
B5% qf his employees “excellent" and the remaining 1?% 'Vary 
good " In another department the supervisor may set a very 
much higher standard of performance and rate 85% of his em- 
ployees good and distribute the other 15% anywhere along the 
Kale from excellent to unsatisfactoiy. In attempting to solve 
the problem of maintaining tu*sonable uniformity of seiyite 
ratlpg standard* 'tome jurisdictions— notably; the United State* 
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The “San Francisco System" of service rating was put into 
effect by the Civil Service Commission of San Francisco oo 
July 1, 1944, for probationary employees, and shortly will be 
applied to nearly 15,000 employees of the County and City of 
San Francisco. Before the system was developed and adopted, 
a comprehensive survey was made of various systems in use. 
This paper is concerned with a presentation of the findings of 
our survey and a description of the approach finally taken in 
developing the “San Franctseo System." 

Our research took many months, covering most of the major 
federal, state, and municipal peraonnel jurisdictions throughout 
the United Statee and Canada We received the finest sort of 
co-operation almost uniformly, and in our analysis of the differ- 
ent systems m use, criticism is not intended We analyzed each 
in relation to the specific problems facing us in developing a 
rating system that would be of the most value for our own local 
jurisdiction 

The problem of rating employee performance is still m the 
experimental atage and, up to comparatively recent times, rat- 
ings were considered more or lose unreliable Personnel agen- 
cies have lately devoted a very large amount of research to the 
problem, while leading personnel writers and publication* have 
exhaustively gone into the field 

From San Francisco's viewpoint, tha purpose of inaugurat- 
ing a service rating system was to develop a means of attracting 
more capable worker* into public service as a career, raising the 
caliber of city peraonnel, m bne with the merit system trend 
to further eliminate the “spoils system" from public service. 
Such a service rating system was authorized by our city charter. 
m 
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Housing Authority, the Tennessee Valley Authority, and the 
New York State service— have adopted a plan based on the 
idea that the categories of “excellent" and "unsatisfactory" 
should be reserved for those employees who dutlnguieh them- 
selves from their fellow worker* unmistakably, to either the 
advantage or the disadvantage of the service, and when such 
ratings are given, the euperviaor is required to cite on the back 
of the rating form, or on a special form, concrete evidence of 
such service 

Fourth The rater should take the ratee into his confidence, 
show him his ratings, and discuss them with him When the 
employee knows what traite or qualities ere judged to be un- 
satisfactory, his normal resetipn would be to seek self-improve- 
ment Especially will this be the case when rating! are used 
os a factor m promotional examinations, and also in determin- 
ing the order of layoff and reemployment, and eligibility for 
salary increases, transfers, leavse of absence, and other privi- 
leges. How the employee does hie work must be determined 
in most -cases by the employee's immediate supervisors One 
of the problems therefore ia to secure reliable ratings from vari- 
ous supervisory officers. In this respect there are two schools 
of drought. One maintain* char the actual assignment of a 
rating should be made by the supervisor! who are direfctly In 
charge of the employee! The other and more recent school 
confine! the supervisor to the reporting or significant behaviors 
or activities, relying for the actual valuation upon some me- 
chanical scoring system administered by the central personnel 
Staff 

1 In devising a rating system the two principal technical 
problem Si have to do with the selection of factor* to he rated 
and the comparative evaluation of ouch factors. Authorises 
generally are agreed that the objectivity and reliability of rat- 
ings Increase as the factors involved become more specific. The 
chief fault to be found with rating plana m the past Is that aticfi 
plana provided for over-all ratings by rim supervisor. Erevan* 
practice is Opposed to reliance upon suttfi highly 'subjective esti- 
mates. It prefers, overwhelmingly, raritigs which, are objec- 
tive. The maximum degree of objectivity Is approached if the 
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attention of the supervisor u focused on specific job behaviors 
or activities, In addition, each item to be reported or evalu- 
ated should bo clearly described in simple, unambiguous terms 
and as concretely as possible 

The point should be stressed, however, that no matter how 
carefully factors have been selected and no matter how care- 
fully supervisors observe and report the behaviors of their 
employees, the whole system will break down unless the actusl 
rating baaed on these u sound Almost without exception, in 
the opinion of Mosher and Kingsley, the devices that have been 
developed to evaluate employee services have failed to provide 
an adequate measuring instrument. 

One method of rating the efficiency of employees, especially 
in industry, is through the use of production records. Such 
positiom u stenographer, typist, file clerk, machine operator, 
Dr copyist lend themselves to unit measurement. In some of 
the larger federal establishments, such as the F arm Credit Asso- 
ciation and the Fedaral Reserve Bank of New York, the num- 
ber of pages typed, the number of errors, and the general 
appearance of the work are taken into consideration by super- 
visor! when typists art rated. 

Another method is through the use of periodic teats. This 
method hw been extensively employed in tome of the federal 
bureaus for rating the efficiency of their employees. In the 
Department of Justice, for example, the service ratings of 
stenographers and typists depend in pan upon their standing 
in periodic speed teats, while performance on a periodic sorting 
test is one basis far determining the rating of postal clerks 
This method-of rating, however, is adapted only to routine and 
repetitive jobs. 

The method In most general use is that of rating schedules, 
A description of some of the schedules in use id other jurisdic- 
tions* follows 

]. Tha Graphic Rating Scale, Thu, according to many 
personnel experts, u the most popular rating method in use, 
being widely employed in private concerns as well as by a num- 
ber of public personnel administrations 1c consists essentially 
of two elements; (1) a list of traits or activities arrived at by 
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an analysis of facton leading to iucku or mating [„ 
on the job, (2) various dwmptive phnaea or adjwjya 
ing the several degrees of e particular activity ot trait. Tvl 
form of the device is aa follows. 
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The descriptive phrkses serve as a guide to the rater who b 
instructed to place a check mark along the line b one of the 
10 boxes above the phrase or between the phrases at the point 
which, in the rater's opinion, represents the degree of the par- 
ticular quality possessed by die ratee. 

This method is open to the objection that there is no stand- 
ard unit of measurement involved and the device is not really 
a scale at all. It is a schedule tha various item* of which are 
arbitrarily weighted and given a numerical value though the 
use of a scoring stencil The device is scored as a rule by the 
application of a ten-point stencil to the straight line, 

2. The Probst Service Rating Sysiem, This system, dwtj- 
oped by J B. Probat, the Chief Examiner of the St Paul Civil 
Service Cbmmianon, in 1928 was in the opinion of personnel 
administrator! the only system which up to that time showed 
any real marks of merit. It has been tnod out in a number ol 
civil service jurisdictions including the Cincinnati, Detroit, Lot 
Angeles, and California Sure public services. Mr. Prohit dis- 
carded the scale idea previously mentioned and substituted for 
it a list of about 100 characteristics or modes of behavior The 
rater checks only those items which are known to describe the 
r Stee| the rest he leaves blank. In addition, he is not required 
to measure relative degrees of a quality m the ratee Accord- 
ing to Its author the scheme is so designed chat failure on tha 
part of the scoring official to correctly and conscientiously check 
the employee's traits can be ascertained at a glance. It hi! 
the added advantage that elaborate instructions to reporting 
officers are unnecessary. 
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The actual evaluation ol .employee service is made by the 
use of a mechanical scoring lyatcm administered by the centre! 
penonnel office. This scoring system hni been one of the prin- 
cipal bases of atticism of the plan because of the difficulty in 
understanding it. 

3, California Report of Performance Plan. In 1938 the 
California State Personnel Board abandoned the Probst Sys- 
tem then in use for one which, in the opinion of the Board, more 
satisfactorily and adequately measured the performance of 
State employees. A separate report sheet for different types 
of work containing a particular combination of work character- 
istics was developed At the preient time 45 different report 
forms each containing a separate list of facton to be rated are 
employed, There are five posublc gradations of markings for 
each item, represented by five column! There columns are 
defined in terms of the extent to which an item is characteristic 
ol the work of an employee. The individual in highest author- 
ity who is in intimate contact with the work of the employee 
acti as dio “reporting” officer. After the reporting officer has 
prepared the report he must review it and diicuss it with the 
employee, who is also given a copy ol the report. The original 
is sent to the personnel agency for scoring. An appeal board 
baa likewise been set up to hear and decide cases in which the 
supervisor and the employes disagree on the markings of a par- 
ticular report. 

4, The Lot Angeles City ScHoolr recently adopted a per- 
formance report system of the trxit-ratuig type where four 
separate forms qre uied. A unique feature of this plan is that 
It was developed by representatives of employee groups and 
organisations. These groups studied and discussed the prob- 
lem for about a year before submitting the final plan to the 
personnel board. The forms are m compiled that they may be 
machine scored- These two features, end especially the first, 
rpight well be given favorable consideration if and when a rating 
lyetem for a]l employees u put into effect Employee support 
would thus be assured, without which. It u claimed, no plan can 
fueceed. 

5, The Spa Francisco public schools are at present using* 
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racing form for non-certified personnel, It consists of a three- 
step rating schedule in which, three degrees of each trait corre- 
sponding to excellent, good, 1 an<f poor are identified by appropri- 
ate descriptive phrases arranged in three columns Such a per- 
formance report fa open to the objection that the ratings are 
too broad or, in other worda, that they do not give a sufficiently " 
fine graduation of the relative excellence of the employees, 

6. The Home Owners Loan Corporation utilizes a service 
rating form that consists of three traits which are rated directly 
by the rating officer on the bans of excellent, very good, good, 
fair, or unsatiafactoiy. This method la open to objections, 
among them these! the tnite to be rated are too few to yield 
a comprehensive picture of the employee's ability, alio, the 
traits are compound ones and require an over-all rating of more 
than one type of activity, 

7 The City of St, Low, Department of PetsonAel, very 
recently adopted a service rating plan according to which em- 
ployees are rated on eight specific traits and an over-al] evalu- 
ation of work performance. These nine measures are felt to 
include all facton necessary tp arrive at a comprehensive evalu- 
ation of employee performance and to apply to all types of posi- 
tions. Thu plan, like that of the H 0 L G, is open to the criti- 
cism that the rftings are too subjective since the tnuts are 
rated by two different supervisors who place a check msrk. 
In one of five columns headed excellent, very good, good, fair, 
and poor Tha subjectivity of the ratings is lessened to some 
extent, however, by a provision whereby the supervisor u re- 
quired, when a rating of either excellent or unsatisfactory on 
the over-all evaluation is given, to furnish substantiating evi- 
dence dF the employee’s superior or inferior performance on the 

job. 

8 The Detroit municipal service rating report consists qf 
a three-step rating schedule in which degrees of poor, satirist' 
toty, and above average of 25 traits are rated objectively with 
the aid of descriptive phrases which identify the various degrees 
of the trait Ten additional traita are rated where the employee 

is in supervisory charge of other employees' work. It sppMJf 
to have considerable merit. The number of traits rated objefr 

* f 
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ij^jy ]f sufficiently large to give a comprehensive and reliable 
pkiine of the employee's worth. The one criticism that might 
Reveled at the method of rating is that only three degrees of 
trait are rated in place of the conventional five-step rating 
Khedule- 

9, The Minnesota State report of employee performance ia 
u adaptation of the California State employee performance 
jam, It conatata of a large number of traits (about 60 m all) 
winch may be measured objectively in terms of the frequency 
wuh which these traits can be observed The traits are checked 
a one offive columns Different forma are used to rate differ- 
<at gmupa of employees. Complete instructions for use of the 
ptmg foim are contained on the reverse side of ,the form The 
[ugo number of traits to be rated makes the scale rather cum- 
bupMne; it has the merit, however, of giving a very compre- 
tailve picture of employee worth 

10. Saginato, Michigan, uses an employee service report of 
die graphic rating scale type Eight traita are raced objec- 
oveiy with the aid of phrases descriptive of degrees of the trait 


irorth m the opinion of the met The form has the merit of 
ntrtme simplicity and allows the rater wide latitude in his 
jppjsual of the degree of each trait possessed by the racee. 

11 A distinct departure from the conventional rating 
schedule in general use is the “Employee Guidance Sheet," as 
it is called, recently developed by the Alabama State Personnel 
Dtpertment, The horizontal scale has been discarded in favor 
of a check-list arrangement The five descriptive phrases 
identifying the various degrees of the ten traits to be rated are 
ouched in language intended to help and encourage the em- 
ployee rather than to report findings in a coldly impersonal and 
noenmes quite blunt manner. The following sample will illus- 
trate the effort that was made to humanize the* report and 
ittmulate the employee to greater effort 


Usual Form 

Quantity of work. 

! ) Unusually high output, 
i High output. 

1 Normal output, 

| Limited output. 

) Insufficient output, unsatisfactory 
Alabama State Form 

! Quantity of work 

Just a friendly suggestion, 

) Exceptionally high output. Keep it up. 

) Better than avenge. Good going. 

) Meeting our requirements, 

) You could do more Try harder. 

) You muld do a lot more. Try much harder 


Other features include a statement in which the repotting officer 
indicates whether the dutiea of the position have changed since 
the last rating period and also a scale on which the department 
head indicates his opinion of the supervisor's ability as a rater, 
All in all, this rating form seemed by far the best of any studied 
by ut. It combined all the features of an exceptionally well 
developed rating system, including simplicity, ease in scoring 
and rating, and high objectivity and reliability. In addition, 
the language employed in the rating scale was so chosen that 
the employee would feel that the criticism was offered in a help- 
ful mood and in consequence should have a strong desire to 
improve his performance if it were below normal. 

The foregoing discussion is devoted to a review on a very 
limited scale of representative rating plans in use None of 
these plans are restricted to the rating of the probationary 
period alone. In inaugurating the San Francisco employee 
rating plan, therefore, it was felt desirable, in order to acquaint 
employees and employers with the idea gradually, to limit it to 
probationers firat The San Francisco charter provides that "at 
any time during the probationary period the appointing officer 
may terminate the appointment 1 * Since power of dismissal is 
vested in the appointing officer and since hn decision is final, 
we felt that simplicity should be the determining characteristic 
of any rating plan for such employees Keeping in mind the 
objectives that should be inherent in every good rating device, 
we attempted to develop a plsn which included the best fea- 
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hues of those reviewed, with particular emphasis on the follow- 
ing points* 

1 The development of a uniform report sheet containing 
traits which wQl fit all types of work. 

2. Clear and concise wording of the descriptive phrases 
which measure the degree of each trait 

3 Phrasing of steps so as to compel supervisors to give care- 
ful consideration to their markings. 

4. A sound and easily applied scaring formula. 

The San Francisco rating plan utilizes a simple check form 
applying uniformly to all similar classes of positions in the vari- 
ous city departments, with the exception of the Poliee and Fire 
Departments, for which separate forma have been prepared. 
Hid form is filled out by supervisors and double checkin) by 
department heads or appointing officers. 

The traits included in the form which apply to all employees 
ire as follows Promptness, Attendance, Quality of Work, 
Ability to Learti, Co-operation, Dependability, Judgment and 
Initiative Additional traits which apply only to certain jobs 
ut' Volume of Work, Contacts with Public, Physical Fitness, 
Appearance, and Initiative In the caae of supervisory posi- 
tions, Ability to Tram and Organization of Work are included 
The five phrases descriptive of the degree of each trait are 
arranged in the form of a check hat, and excellence on the scale 
it rated sometimes at the beginning and sometimes at the end. 
Such an arrangement increases the probability that the rater 
will pay full attention to the descriptive phrases. 

On the back of the report is space For the certificate of the 
reporting officer and of the employee involved. The appointing 
officer in case* involving probationary employees is required to 
indicate his impression of (he employee’s General Fitness to 
bold the position, and to indicate presence or absence of unde- 
sirable characteristics which would make the employee unsuited 
to the particular job. If the probationary employee is rejected, 
the appointing officer must indicate specifically why. Ratings 
w given probationary employees at end of the second and fifth 
months of service, and give appointing officers a legitimate 
erase for rejection of an unsatisfactory applicant by refusal 
to certify him for permanent tenure, 
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The probationary penod is a definite part of the examina- 
tion and testa those factore for which no adequate written or 
oral tests have been devised Unfortunately, however, appoint- 
ing officers often fad to recognize their responsibility in cheeking 
the probationer's services and too often the probationary period 
elapses without adequate investigation of the employee's per- 
formance on the job. It is hoped that the use of the service 
rating system for probationers will serve to impress upon the 
appointing officer his responsibility in determining whether or 
not the probationary appointee wdl be a satisfactory permanent 
employee. 

We expect, for permanent employees, to rate once or twice 
per year 



NEW DEVELOPMENT FOR FIRE MOTOR DRIVER 
EXAMINATION | 

' WILLIAM E. TRVOQ, JR. 

Kmui Cirj pMwaafld Department, Emm City, fcfiunuri 

The development of a new performance testing procedure 
for, Fue Motor Driven followed some extensive research by the 
Personnel Department of the City merit system of Kansas City, 
Missouri This examination is based upon a standardized test 
used by the United Stated Army Engineers to examine all moeor 
vehicle drivers, and was developed by Amos E Neyhart, Ad- 
ministrative Head, Institute of Public Safety of Pennsylvania 
State College and Consultant to the American Automobile 
Association. The Army examination consisted of a short 
written exercise followed by testa for visual sciiity, field of 
vision, depth perception, and the applicant’s reaction to simu- 
lated traffic situations The Army performance procedure in- 
cludes a dosed area and a general road teat, with an appraisal 
of the general driving characteristics of tha applicant. The 
driver’s performance test developed by the Kansas City Per- 
sonnel Department is based pnmanly on the closed area pro- 
cedure used by the Army Engineers 

Five tasks were selected (1) Driving on a straight line- 
forward and backward, which consisted of lining up the front 
and rear left wheels on a painted 100-foot line, and driving at 
normal speed, to keep both front and tear wheels on the lure 
for the entire distance (2) Gauging apace whin steering in 
close limits. This is a timed test m which the contestant has 
to make a 90-degrec turn going forward, and thtn to baek up, 
making the same turn (3) Stopping the car smoothly m 40 
feet while gpuig 20 miles an hour lanes are painted on the 
street as a guide to begin slowing down and to stop. (4) Stop- 
ping the car with the from wheel exactly on a cross painted on 
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the Street This procedure is repeated for the 
and also far the rear wheel and bumper. ‘ The 
penalized doubly for going beyond the line. (5 
ear against a curb in a regulation parallel parking 

A road test through regular traffic conditions includes an 
appraisal of the applicant’s making a nghc- and n left-hand 
turn, of his starting from a standstill while on an upgrade 
finally, of his handling of the car and of his own reactions' and 
emotional status in meeting traffic Bituatiuns A regular tour- 
.mg car was used Tor this first test, but there is no doubt that 
fire department apparatua would providu a much more dis- 
criminating result It was found that generally the routing 
car proved easy for everyone to maneuver through the various 
operations 

One of the heat developments of the Neyhart testing pm- 
cedure n the detailed scoring sheet devised for the use of the 
examiner Under each task are outlined several items upon 
which the contestant is to be graded, e,g. 

Sample Score Sheet 

III Stopping smoothly in 40 fact at 20 miles an hour— Front Dunum 
• Driving Forward through Stanchions, 

1 Moves gear shift lever to another position without dashing 

2, Keeps an even speed —20 M P H 

3 Moves vehicle continuously — no stops 

4 Steers With certainty— no sudden jerks 

5 Does not it op between stanchions 

6. Does not hit right— left stanchions 

7. Does not race engine, 

8 Does not stall engine. 

9 Stops vehicle smoothly. v 

10 Stops front bumper snort— over line 

A marc accurate and discriminating grade is made possible by 
this system, and the driver may lator see the points of the exer- 
cise that he failed to perform correctly. The examiner checks 
those points missed as the operation is being completed and 
scores each exercise before going to the next It is believed that 
this has proved a much more objective test than others, and by 
the ime of the detailed scoring sheet, the standards outlining the 


front bumper 
contestant a 
) Parking the 
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coune, and Itnca drawn in the street to guide the contestant, 
the result depends lets on the j’udgment of the examiner and 
provides a positive, equal standard for all applicants. It leaves 
little room for the driver to dispute a point, as there can be no 
doubt when he hita or knocks down a standard, 

The equipment ui«d for this teat is an important factor, 
both for the applicant and the examiner Standards painted 
white and topped with red flags are used to outline the differ- 
ent courses, supplemented by four-inch white lines painted on 
the street as a guide to the dnvar The tasks for this test may 
all be lai^ out m an ordinary street 

The examination far Fire Motor Driver is divided into four 
sections, a written examination, performance test, service rat- 
ing, and rating of experience and training. The written and 
performance sections are both given a valuation of 30%, -with 
the service rating and raring of experience and training each 
being valued it 20% of the total, The service rating is also a 
new development and was used for the first time in this exami- 
nation* There has been no further opportunity to determine the 
validity of this new procedure, except for the correlations which 
have been computed. Following are the correlations between 
the various sections of the test I Performance and the total 
score, +18; performance and written, +.02; written and total 
score, performance and service rating, - 17,* The low 
correlation between the performance test scores snd written 
scores is regarded as particularly desirable, since the two sec- 
tion* are testing fat different qualities. 

The swing and grading system for the performance test is 
baaed on a taw score of 100. The test is divided into ten tasks, 
each bne having a valuation of 10, Points are deducted for 
each Item of the operation that was performed incorrectly. A 
'distribution table, based on fhe scores, u made up, and a pois- 
ing p3lnt is set A conversion table u computed on the basis 
of the 30% valuation ret for the performance teat The total 
raw score is then converted and added to the rest of the score 
(or the enure examination. 


* film ihfa h me ol ths i«rriM mini on is aovbniicn, w* had 


The Army Engineer* and the Fire Department worked m 
full cooperation with the Personnel Department in developing 
the examination. With some further refinements, it is believed 
that this will constitute a highly valid and successful testing 
procedure 
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r«Uif. 8 . "A Bint 17 d Taro lor the Dora mint Eye." Journal 0 / Canard P 17 - 
Oden, XXXI (1944), IIV-ML . . , 

la order lo meet the need tor uiuforeutv of procedure! ind tat mite rule used 
b dnenron* eye damminw, the author Ku developed 1 battery d unn iimple can 
(m ihe puroou. Directum lor miking the tat milenali, precne inetrucuuu lor 
■fmimitniiog. 1 form lor recording pertotraince, and norma bawd an the rttwdi of 
— t, mn nbltru an pm The auibor reportt that "coeBicKna of nliabdity deter- 
by repealing the tail an all over J9S or 39 " Of the pa [Hill boo lor which 
-gnu are given, S3 per cent wen definitely ngbt eyed on the bit toy, 24 per cent 
mi jrfituitJt' Ida ored, 7 par cent Impartial -eyed, and rha remainder thawed loft- 
rjti tendency or ngfawyed tendency BJtik S /ay 

By idecttni portion d the EaiAiid lo* r* PkjnCl Ap 1 

Tut 0 / Anihniilt Rt*iord«i, the BnnrU Tttl 0 / Uachmleal Comyrvirarfoft, the 
j iattiNtll Exomnatam, and o their, an aptitude UR wit entutructad for me In 
(uailoi iiutablllry of eandldaca lor tmnuif tn techmcal work oa the high-*choal or 
cdlric lirb- The na piru of the examination can bo admrantund In Ti mlautn 


ileal Sckjic* Aptitude Tat” 
-3B7 

ApUtuda Tat I, the Moot* 


Phwal Sdenee Conrprchciunn, Anihmotie Baton! ni, Verbal Compnhttiloa, ltd 
Uatfiinlcxl Cnropnbeailan, repmeonoi lelitivdy 10 dep oldest aipecu of technical 
pofornarcf. Edith & /ay, _ 


Hoi linear. t I “The Refatmnahlp bitwaen the Centroid Maihod and Sparman’i 
MclM." Journal 0 / SJutMlbmtl PtycAokgy. XXXV (1944), 347-312 
fib article polflli out the relillomhlp hatwten the centroid method of obtaining 
furor roeffideou and Spearman’i 1914 method, 11 wdl aa the rdatnntbro between 
dtf reniroid method inn Spaimsan'e theorem on the condition of romi Itlathown 
(hit rf eommtinallua ire liuened in the diaiuoala, Spnrmer'i method lor oblilnm* 
1 factor coefficient n the tame at the centroid method Simflarly, the method of 
conflation of luma, anio with the cumroupalma Included, w oqurvalent to the 
centroid method, it bone damonimtrd that "cm trend codflriton a/ may thsa bo 
mirdid ai carrelatifxu between a v» ruble lj and the total (or arcrtie) «d the 
rwiabla «!,■*,«• projected on the common -ficmr ipxce." L. Bnoihtlu 

Confer, Homer "Blmplifyin( the Scwuil Technique of the BemreUtei Peiamiality 
Inrmtory " Jo.nuJ 0 / ApjJlad Ptytkdon. SxVUl (19M), 412-413 
Five almpitfied keyt were uted u replace the uiual •coring technique of the 
Bimnulrr Prnanahty fmuntoryr I, All original valua of S to -3 ware iiaigncd 1 
value of 0, 4 and above became 1, -4 and below became- 1 II Orifintl valua d 
2b - 2 became 0, 3 and above beam 1.-3 and below beam* -1 III "f° 
gun were ignored mad ell “Yea" and “No" Valua hiVrnf ■ difference of ttvta 

' Edited by Fort cat A. Kmgibury 
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(reduced to Eve oa icile B3-1) or more were paired. Fontive nine* btneu I and 
pegaLn valuta bmgu-1 IV “I" rnpoma were i« norad and ill paitive man 
wen coqb ted 11 1 i then wn ■ differenei of ail (reduced "o five On mle BM) or 
n “ Te ■ba la" and "No" an* we re, V All pontm rupotiHi were counted 

u I Zero and Mjatim wtro ignored. After the now anna were derived. Puraon 
eorrejattoni were worked for each real* with reinlia from each key Seventeen id the 
rarrelatniu or the Bra ktya with oniioal percentile ranki on the four Kiln ram* 
fro® 02 tn 91 Probable errore rente from t-Olf to ± 032 

It a concluded that the but new key lor each tint could ba turd for quck loci 
ikm of extreme cam, the middle ifty per cent tonaldered “normal," and the Mty per 
rant M tbe two eairemn rocored With the rewular Danwenur krya. The row key* 
effect ■ tivina of moie than 90 per con la leering tame. Aferfae R CmjD 

Lund in, R W “A PrcUmlnuy Report on Some New Tnl* of Muncal Ability” 
/ouraai 0! A tflUd Piytlalory XXVIII ( 1944). 19J-39S. 

Tha doc riba ■ ureay of 1 mu dev bed 10 nuiure directly and objectively 
anwal aURla, lamed rather than utnate. The tern— recorded moncal iiemi cm 
whim use or different" jndjmrata mre requated for iBUrval dHcnnutuuoa, 
adoaie tnmpaertnn, hirmotnc iruipaitmn, melodic tequenea and havnunm aa- 

n au-wera mrca to 2 mupi of lubjeeu 60 mnnc iwdenti from DtPauw and 
ana Uolraiuna and lin itudtatt from tmdcrireduau dnua in mychakwy at 
(he Inter The total icttm (hawed 1 fallibility of 72 , hut the refialnlitra 01 the 
lepaniB tan were not ao huh. The only ernenon for validitiBi wn I be auiho 1 
epecuily eunuructed jpiphic raRnt ictla by whkb the profciaon of muac traded 
their irudcoo. Pindua a eitruEnnt diffeince between the 2 poupi, the aoihor 

belwrax Ihc 1BO a- JS a a a — * — L-“- . -aa — 

Far eon S Tnthl 

McCarthy, D "A Study d the Rdfabduy of the Goodrooueh Diswlor Tat d 
Iordlifeisc*." Jonntl of Pryckclvn, XVIII (1944), 201-7i& 

The Good«MU«h Drtwilt Tut of /el<iff|ixrf wai area two time*, a week 
■pair, to 306 third and fourth-arede chidrea Eacb int wai icurcd three tuna, 
twice by U 10 unit leorer, and once bv different leorera. Tan icored by |h> um 
,, perton lira * corrdatloo of 54, with 1X4 par cant of the caia havint a diarrepancy 
of one year or roore Condition betweoi Kora by different mnn waa 50, but 
dberapanaM of ana year or more were found In 21 3 per cent of the tan More 
aver, alihauth the conintency on two lata token at adftrenx tima and icored by 
one pinu wai .40, there wax t duerepaoey ef at Lau obi year In 41 7 per rent M 
■he naea. The odd mu relubility computed with the Spearman Brown prophecy 
foiRndi wai Bs. The rauln, danonuritlot both the nbjacavny of icortnj[ and 
the nnabdiiv 16 performance over a abort interval, ehow the Mod for caution in 
tnmj the icile for mdmdual dugaou*. L, ffouikifaL 


McCtedaod, David C "Simplified Bcormi of iha Bcrareutei PoMnality Invnlury 
/annul 0 / A f M p, ycAofot, XXVIII (1944) ,414-419 
The icpnni a tha Bmrtuur Ptrmulltv /nwaloiy wai ilnplifiad by anitaht 
n wrighied 3 or ibovx and ■ value d 
wen wr 11 bled berwetn that I Irma to 


ilnplified by anigntnt 
1 d - 1 to all Kuweit 
were itnonA Corn- 

iinoni orewini mu atm lunpiinen mmiq 01 toe utvemone* of 1 14 edlcsa men on 
five tram were computed, they were above 95 lor BIN, B4D PIC, aoa FIS, and 
M (or B2S. Formulu for con vara og a ihort aeon into ■ foil kotc and 4 tide d 
ihort icon pcrctoule oorero for eolleye men are Included. The time required to 
A — 1 — 1 * — Afonon U Crtnui 


Mellon*, M A “A Fit tonal Study of Picture Tata lot Young Children." Bnluh 
Journal 0} Pathology, XXXV (1944), 9-16 

A battery of 14 datura tvu, a word readme tea, end ■ mrchxnln! imhmcuc 
■ ^ ind 196 firia The 
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factor promt (a die ampfere battery but not m ihe picturo-icin battery alone. 11 
()) a Ijrecc factor, appearing m mine of the picture taau, wfatch ihored a tea differ, 
tore, evident ody bt the boyt L BootMtl 


Oddi, C, W "The Scoring of Continuity or Rearrangement Treta." Journal of 

kiueulorul Pwkclan, XXXV (1944), JSJ-JJ6. 

Tata in which the talk 1 * 10 rank up to im ft tore in iron* deaigpated order can 
be eared quickly and 10 ■ theoratkiUriound manner by tiring a tabic presented aa 
an aU |n ehonmlnf tha cakuladqqi. The ubla u divided into ucimo* which eiubl* 
the Korer m pcniliu with imma aaoia lor poorer than chanra areangemont, m give 
eero Kora tn all airangonenia mgtnvdy correlated with the correct order, or to 
aeon by ih* better, but longer, y method which require* that difference* between 
Brest rank* and popil'i rupture «* be iquared and luramad. Edith 5 Joy 
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dent Permuid Work held at the Uelvenlry of Calffarnia, Lot Angela, Summer 
Scukm, 1944 Publuhcd by Wetter 0 Pcnonnd Seme* Paiadeni, Caldtunn, 
1944 

Tbia M u 20-page pamphlet prepared by a comm lure at ihe lemmata It aim 
to proenr the different pamn of view fat a lent* d ihort article*. E. G, WOlunuon 
u ''Leader of Ihe Ini ntute" emphiena ihe need for a we]) pfawert miegmtd per- 
tonnd progtam. The tlnfBunco d “the contnburemi of the iludenu ibeoHelva 
to then own development" a ducuted by Jen If E, Gdwoo, "Dun of Wronen." The 
"PctManel Admmmntor," Karl W On thank. In reviewing the 
at the metlina bimge out many of the pnqilnl coroidentliuw 1 

iha war F T Perfci — — “ " 

tny ‘ 
to fa 


Tho 

. ipere read 

that mure bq mot in 
icnta favorably on 
*ff OM Ruling, but 


Imprtued by d» evidence 

n the pQitwnr yean, but that the new proleu Ion 
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Rabin, A, 1 "Tm Cotutuvy and Vanaum fat the Muallj IR" Journal of 
Orareol PiycAoieip-, XXX I (1944), 231-339 

The W*akiUr-BtU*vu4 lnltSig*a£a Staltr wen uted on 60 New Himpehjn State 
Horn real adult pinroti to determine the rdativo tnt-ictm cotuouncy of mental 
paiienn a* well ■■ the lerolilrtty of the mtuurme numimaoL The rualta reveal ad 
a cunditKM of A4 between them and compared favorably with ihoto obtained from 
different wax* oa normal ptnooe. tending to diaprora the belief that tux Tenia of 
nydioaa are wry mutable. The coefficient aim Indicated a high degree of rtju* 
btfur (or the whole teal* itaeff, icatnUng to the auihor’e cucdoaiotii. Slight but 
cbuutem imwn In total and lodividoiT retat ecore* were obemrad, apedally on 
aha nerfoimuca pare d the mle, roe mags l rude being directly related re the extent 
of rime between tat end treat, Perem S. Track t, 

TueknunjJxcsb. "A Study of the RdnbSlty d tbe Mbnaota Rate d Manipuli- 
Ikm Tot by the MU-H«H and Tat-Rnut Mtthodx." Journal of AfpBrt 
SwdWogy,i3CVHT 0944) . 388-J92, 



poaible core bin 1 me 1 of iingl* tnifa range from 82 to 4. 

du a|uil cot and iha lean nml Irani to 14 day* rath a iwdun of 7 day*. 
the ten retat imdy • prononvc though imall decreaia in tun for each roceeoiva 
trial for both Pladog ind Turning on iba initial and final tat u remind, AR *oh- 
lecti wav faater u the retat for both Placing and Turning. The D /0 difference 
between total acorn a imifal and final tat a 6.7 for Placing apd 10,4 Tor Ttarohg. 
hlorlon H Crauj, 
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